Re: [dm-devel] [PATCH v2 13/21] libmultipath: provide defaults for {get, put}_multipath_config

2020-09-24 Thread Benjamin Marzinski
On Thu, Sep 24, 2020 at 03:37:08PM +0200, mwi...@suse.com wrote: > From: Martin Wilck > > Add an implementation of get_multipath_config() and put_multipath_config() > to libmultipath. The linker's symbol lookup rules will make sure that > applications can override these functions if they need to.

Re: [dm-devel] [PATCH 10/11] libmpathpersist: add linker version script

2020-09-24 Thread Benjamin Marzinski
On Thu, Sep 24, 2020 at 03:36:43PM +0200, mwi...@suse.com wrote: > From: Martin Wilck > > This defines the ABI of libmpathpersist in the current state. > --- > libmpathpersist/Makefile| 6 -- > libmpathpersist/libmpathpersist.version | 20 > 2 files chan

Re: [dm-devel] [PATCH 00/11] multipath-tools: add linker version scripts

2020-09-24 Thread Benjamin Marzinski
On Thu, Sep 24, 2020 at 03:36:33PM +0200, mwi...@suse.com wrote: > From: Martin Wilck > > Hi Christophe, hi Ben, > > Patch 1-5 are small fixes, the first two resent from an earlier > submission. Patch 6ff. add version scripts for the linker to > libmultipath, libmpathpersist, and libmpathcmd. >

[dm-devel] Make LUNs higher than 255 more friendly to look at

2020-09-24 Thread Brian Bunker
For LUNs between 0 and 255 peripheral addressing is used. For LUNs higher than 255 the LUN addressing should switch to flat according to the specification. Instead of printing out the LUN number without regard to the shifting of address method, display the LUN as it was intended to be the user c

Re: [dm-devel] [RFC PATCH 0/2] dm crypt: Allow unaligned buffer lengths for skcipher devices

2020-09-24 Thread Damien Le Moal
On 2020/09/25 4:14, Sudhakar Panneerselvam wrote: >> >> On Thu, 24 Sep 2020, Sudhakar Panneerselvam wrote: >> By copying it to a temporary aligned buffer and issuing I/O on this buffer. >>> >>> I don't like this idea. Because, you need to allocate additional pages >>> for the entire I/O s

Re: [dm-devel] [RFC PATCH v2 0/3] add library to check if device is a valid path

2020-09-24 Thread Benjamin Marzinski
On Thu, Sep 24, 2020 at 07:22:21PM +, Martin Wilck wrote: > On Thu, 2020-09-24 at 11:30 -0500, Benjamin Marzinski wrote: > > On Thu, Sep 24, 2020 at 08:18:02AM +, Martin Wilck wrote: > > > On Wed, 2020-09-23 at 23:59 -0500, Benjamin Marzinski wrote: > > > > > > > > The second patch adds ge

Re: [dm-devel] [PATCH 08/11] libmultipath: create separate .so for unit tests

2020-09-24 Thread Benjamin Marzinski
On Thu, Sep 24, 2020 at 03:36:41PM +0200, mwi...@suse.com wrote: > From: Martin Wilck > > The unit tests use a lot of internal symbols that we don't want > to add to the ABI if we don't have to. As long as we don't > have to make incompatible changes to functions, we can work around > that by sim

Re: [dm-devel] [PATCH 23/23] multipath-tools: mpath-tools.supp: file with valgrind suppressions

2020-09-24 Thread Benjamin Marzinski
On Thu, Sep 24, 2020 at 03:40:54PM +0200, mwi...@suse.com wrote: > From: Martin Wilck > Git complains about the extra blank line at the end of the file third-party/valgrind/mpath-tools.supp:33: new blank line at EOF. -Ben > These leaks are caused by other libraries (libsystemd, glibc, > libgc

Re: [dm-devel] [PATCH 06/11] libmultipath: checkers/prio: allow non-lazy .so loading

2020-09-24 Thread Benjamin Marzinski
On Thu, Sep 24, 2020 at 03:36:39PM +0200, mwi...@suse.com wrote: > From: Martin Wilck > > If compiled with -DLOAD_ALL_SHARED_LIBS, all known prioritizers > and checkers will be loaded immediately on startup. This is useful > for determining symbol usage (start executables with LD_BIND_NOW=1). It

Re: [dm-devel] [PATCH 02/11] libmultipath: copy mpp->hwe from pp->hwe

2020-09-24 Thread Benjamin Marzinski
On Thu, Sep 24, 2020 at 03:36:35PM +0200, mwi...@suse.com wrote: > From: Martin Wilck > > Since f0462f0 ("libmultipath: use vector for for pp->hwe and mp->hwe"), > we've been trying to fix issues caused by paths getting freed and mpp->hwe > dangling. This approach couldn't work because we need mp

Re: [dm-devel] bdi cleanups v7

2020-09-24 Thread Jens Axboe
On 9/24/20 12:51 AM, Christoph Hellwig wrote: > Hi Jens, > > this series contains a bunch of different BDI cleanups. The biggest item > is to isolate block drivers from the BDI in preparation of changing the > lifetime of the block device BDI in a follow up series. Applied, thanks. -- Jens Axb

Re: [dm-devel] [RFC PATCH v2 0/3] add library to check if device is a valid path

2020-09-24 Thread Martin Wilck
On Thu, 2020-09-24 at 11:30 -0500, Benjamin Marzinski wrote: > On Thu, Sep 24, 2020 at 08:18:02AM +, Martin Wilck wrote: > > On Wed, 2020-09-23 at 23:59 -0500, Benjamin Marzinski wrote: > > > > > > The second patch adds get_uid fallback code for dasd devices. The > > > third > > > patch just c

Re: [dm-devel] dm-raid: stack limits instead of overwriting them.

2020-09-24 Thread Mike Snitzer
On Thu, Sep 24 2020 at 2:56pm -0400, John Dorminy wrote: > On Thu, Sep 24, 2020 at 1:24 PM John Dorminy wrote: > > > > I am impressed at how much I read wrong... > > > > On Thu, Sep 24, 2020 at 1:00 PM Mike Snitzer wrote: > > > > > > On Thu, Sep 24 2020 at 12:45pm -0400, > > > John Dorminy wr

Re: [dm-devel] [RFC PATCH 0/2] dm crypt: Allow unaligned buffer lengths for skcipher devices

2020-09-24 Thread Sudhakar Panneerselvam
> > On Thu, 24 Sep 2020, Sudhakar Panneerselvam wrote: > > > > By copying it to a temporary aligned buffer and issuing I/O on this > > > buffer. > > > > I don't like this idea. Because, you need to allocate additional pages > > for the entire I/O size(for the misaligned case, if you think through

Re: [dm-devel] dm-raid: stack limits instead of overwriting them.

2020-09-24 Thread Mike Snitzer
On Thu, Sep 24 2020 at 2:12pm -0400, Mikulas Patocka wrote: > > > On Thu, 24 Sep 2020, Mikulas Patocka wrote: > > > > > > > On Thu, 24 Sep 2020, Mike Snitzer wrote: > > > > > WAIT... Could it be that raid_io_hints _really_ meant to special case > > > raid0 and raid10 -- due to their stripi

Re: [dm-devel] [RFC PATCH v2 3/3] libmultipath: change log level for null uid_attribute

2020-09-24 Thread Martin Wilck
On Thu, 2020-09-24 at 11:20 -0500, Benjamin Marzinski wrote: > On Thu, Sep 24, 2020 at 08:06:41AM +, Martin Wilck wrote: > > On Wed, 2020-09-23 at 23:59 -0500, Benjamin Marzinski wrote: > > > > > > diff --git a/libmultipath/discovery.c b/libmultipath/discovery.c > > > index f650534f..435f2639

Re: [dm-devel] dm-raid: stack limits instead of overwriting them.

2020-09-24 Thread John Dorminy
On Thu, Sep 24, 2020 at 1:24 PM John Dorminy wrote: > > I am impressed at how much I read wrong... > > On Thu, Sep 24, 2020 at 1:00 PM Mike Snitzer wrote: > > > > On Thu, Sep 24 2020 at 12:45pm -0400, > > John Dorminy wrote: > > > > > I don't understand how this works... > > > > > > Can chunk_si

Re: [dm-devel] [RFC PATCH 0/2] dm crypt: Allow unaligned buffer lengths for skcipher devices

2020-09-24 Thread Mikulas Patocka
On Thu, 24 Sep 2020, Sudhakar Panneerselvam wrote: > > By copying it to a temporary aligned buffer and issuing I/O on this > > buffer. > > I don't like this idea. Because, you need to allocate additional pages > for the entire I/O size(for the misaligned case, if you think through You can b

Re: [dm-devel] dm-raid: stack limits instead of overwriting them.

2020-09-24 Thread Mikulas Patocka
On Thu, 24 Sep 2020, Mikulas Patocka wrote: > > > On Thu, 24 Sep 2020, Mike Snitzer wrote: > > > WAIT... Could it be that raid_io_hints _really_ meant to special case > > raid0 and raid10 -- due to their striping/splitting requirements!? > > So, not raid1 but raid0? > > > > E.g.: > > > > d

Re: [dm-devel] [RFC PATCH 0/2] dm crypt: Allow unaligned buffer lengths for skcipher devices

2020-09-24 Thread Sudhakar Panneerselvam
> By copying it to a temporary aligned buffer and issuing I/O on this > buffer. I don't like this idea. Because, you need to allocate additional pages for the entire I/O size(for the misaligned case, if you think through carefully, you will know why we have to allocate a temporary buffer that is

Re: [dm-devel] [RFC PATCH 0/2] dm crypt: Allow unaligned buffer lengths for skcipher devices

2020-09-24 Thread Mikulas Patocka
On Thu, 24 Sep 2020, Sudhakar Panneerselvam wrote: > Hi Mikulas, > > > > Windows Guest <--> Vhost-Scsi <--> LIO(scsi/target/blockio) <--> dm-crypt > > <--> iSCSI block device > > > > > > One real example out of my debugging: Windows sends a I/O request with > > > 6656 bytes to vhost-scsi inte

Re: [dm-devel] [RFC PATCH 0/2] dm crypt: Allow unaligned buffer lengths for skcipher devices

2020-09-24 Thread Sudhakar Panneerselvam
Hi Mikulas, > > Windows Guest <--> Vhost-Scsi <--> LIO(scsi/target/blockio) <--> dm-crypt > <--> iSCSI block device > > > > One real example out of my debugging: Windows sends a I/O request with > > 6656 bytes to vhost-scsi interface. Vhost-scsi uses translate_desc() in > > drivers/vhost/vhost.c

Re: [dm-devel] [RFC PATCH 0/2] dm crypt: Allow unaligned buffer lengths for skcipher devices

2020-09-24 Thread Mikulas Patocka
On Thu, 24 Sep 2020, Sudhakar Panneerselvam wrote: > Hello Eric, > > > -Original Message- > > From: Eric Biggers [mailto:ebigg...@kernel.org] > > Sent: Wednesday, September 23, 2020 11:14 PM > > To: Mike Snitzer > > Cc: Sudhakar Panneerselvam ; > > damien.lem...@wdc.com; ssudhak...@gm

Re: [dm-devel] dm-raid: stack limits instead of overwriting them.

2020-09-24 Thread John Dorminy
I am impressed at how much I read wrong... On Thu, Sep 24, 2020 at 1:00 PM Mike Snitzer wrote: > > On Thu, Sep 24 2020 at 12:45pm -0400, > John Dorminy wrote: > > > I don't understand how this works... > > > > Can chunk_size_bytes be 0? If not, how is discard_granularity being set to > > 0? > >

Re: [dm-devel] [PATCH 04/13] aoe: set an optimal I/O size

2020-09-24 Thread Martin K. Petersen
Christoph, > aoe forces a larger readahead size, but any reason to do larger I/O is > not limited to readahead. Also set the optimal I/O size, and remove > the local constants in favor of just using SZ_2G. Reviewed-by: Martin K. Petersen -- Martin K. Petersen Oracle Linux Engineering

Re: [dm-devel] [RFC PATCH 0/2] dm crypt: Allow unaligned buffer lengths for skcipher devices

2020-09-24 Thread Sudhakar Panneerselvam
Hello Mikulas, > -Original Message- > From: Mikulas Patocka [mailto:mpato...@redhat.com] > Sent: Thursday, September 24, 2020 6:40 AM > To: Sudhakar Panneerselvam > Cc: a...@redhat.com; snit...@redhat.com; dm-devel@redhat.com; dm- > cr...@saout.de; damien.lem...@wdc.com; Shirley Ma > ; ss

Re: [dm-devel] dm-raid: stack limits instead of overwriting them.

2020-09-24 Thread Mikulas Patocka
On Thu, 24 Sep 2020, Mike Snitzer wrote: > WAIT... Could it be that raid_io_hints _really_ meant to special case > raid0 and raid10 -- due to their striping/splitting requirements!? > So, not raid1 but raid0? > > E.g.: > > diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c > index 56b72

Re: [dm-devel] [PATCH 2/3] dm: add support for passing through inline crypto support

2020-09-24 Thread Eric Biggers
On Thu, Sep 24, 2020 at 12:16:24PM -0400, Mike Snitzer wrote: > On Thu, Sep 24 2020 at 11:45am -0400, > Eric Biggers wrote: > > > On Thu, Sep 24, 2020 at 09:46:49AM -0400, Mike Snitzer wrote: > > > > > Can you help me better understand the expected consumer of this code? > > > > > If you have som

Re: [dm-devel] dm-raid: stack limits instead of overwriting them.

2020-09-24 Thread Mike Snitzer
On Thu, Sep 24 2020 at 12:45pm -0400, John Dorminy wrote: > I don't understand how this works... > > Can chunk_size_bytes be 0? If not, how is discard_granularity being set to 0? Yeah, I had same question.. see the reply I just sent in this thread: https://www.redhat.com/archives/dm-devel/2020-

Re: [dm-devel] [PATCH] dm-raid: stack limits instead of overwriting them.

2020-09-24 Thread Mikulas Patocka
On Thu, 24 Sep 2020, John Dorminy wrote: > I don't understand how this works... > > Can chunk_size_bytes be 0? If not, how is discard_granularity being set to 0? Yes - chunk_size_bytes is 0 here. > I think also limits is local to the ti in question here, initialized > by blk_set_stacking_lim

Re: [dm-devel] dm-raid: stack limits instead of overwriting them.

2020-09-24 Thread Mike Snitzer
On Thu, Sep 24 2020 at 12:26pm -0400, Mikulas Patocka wrote: > This patch fixes a warning WARN_ON_ONCE(!q->limits.discard_granularity). > The reason is that the function raid_io_hints overwrote > limits->discard_granularity with zero. We need to properly stack the > limits instead of overwriting

Re: [dm-devel] [RFC PATCH 0/2] dm crypt: Allow unaligned buffer lengths for skcipher devices

2020-09-24 Thread Sudhakar Panneerselvam
Hello Milan, > -Original Message- > From: Milan Broz [mailto:gmazyl...@gmail.com] > Sent: Thursday, September 24, 2020 2:16 AM > To: Eric Biggers ; Mike Snitzer > Cc: Sudhakar Panneerselvam ; > damien.lem...@wdc.com; ssudhak...@gmail.com; Martin Petersen > ; dm-cr...@saout.de; dm-devel@re

Re: [dm-devel] [RFC PATCH 0/2] dm crypt: Allow unaligned buffer lengths for skcipher devices

2020-09-24 Thread Sudhakar Panneerselvam
Hello Eric, > -Original Message- > From: Eric Biggers [mailto:ebigg...@kernel.org] > Sent: Wednesday, September 23, 2020 11:14 PM > To: Mike Snitzer > Cc: Sudhakar Panneerselvam ; > damien.lem...@wdc.com; ssudhak...@gmail.com; Martin Petersen > ; dm-cr...@saout.de; dm-devel@redhat.com; >

Re: [dm-devel] [PATCH] dm-raid: stack limits instead of overwriting them.

2020-09-24 Thread John Dorminy
I don't understand how this works... Can chunk_size_bytes be 0? If not, how is discard_granularity being set to 0? I think also limits is local to the ti in question here, initialized by blk_set_stacking_limits() via dm-table.c, and therefore has only default values and not anything to do with th

Re: [dm-devel] [RFC PATCH v2 0/3] add library to check if device is a valid path

2020-09-24 Thread Benjamin Marzinski
On Thu, Sep 24, 2020 at 08:18:02AM +, Martin Wilck wrote: > On Wed, 2020-09-23 at 23:59 -0500, Benjamin Marzinski wrote: > > The main part of the this patchset is the first patch, which adds a > > new library interface to check whether devices are valid paths. This > > was designed for use in t

[dm-devel] [PATCH] dm-raid: stack limits instead of overwriting them.

2020-09-24 Thread Mikulas Patocka
This patch fixes a warning WARN_ON_ONCE(!q->limits.discard_granularity). The reason is that the function raid_io_hints overwrote limits->discard_granularity with zero. We need to properly stack the limits instead of overwriting them. Signed-off-by: Mikulas Patocka Cc: sta...@vger.kernel.org ---

Re: [dm-devel] dm-table: set non-zero q->limits.discard_granularity

2020-09-24 Thread Mikulas Patocka
On Thu, 24 Sep 2020, Mike Snitzer wrote: > On Thu, Sep 24 2020 at 10:34am -0400, > Mikulas Patocka wrote: > > > Hi Mike > > > > Could you send this to Linus before v5.9 is released? > > > > Mikulas > > > > > > > > From: Mikulas Patocka > > > > If q->limits.discard_granularity is zero,

Re: [dm-devel] [RFC PATCH v2 3/3] libmultipath: change log level for null uid_attribute

2020-09-24 Thread Benjamin Marzinski
On Thu, Sep 24, 2020 at 08:06:41AM +, Martin Wilck wrote: > On Wed, 2020-09-23 at 23:59 -0500, Benjamin Marzinski wrote: > > If uid_attribute is explicitly set to an empty string, multipath > > should > > log the uid at the default log level, since using the fallback code > > is > > the expecte

Re: [dm-devel] [PATCH 2/3] dm: add support for passing through inline crypto support

2020-09-24 Thread Mike Snitzer
On Thu, Sep 24 2020 at 11:45am -0400, Eric Biggers wrote: > On Thu, Sep 24, 2020 at 09:46:49AM -0400, Mike Snitzer wrote: > > > > Can you help me better understand the expected consumer of this code? > > > > If you have something _real_ please be explicit. It makes justifying > > > > supporting

Re: [dm-devel] [RFC PATCH 0/2] dm crypt: Allow unaligned buffer lengths for skcipher devices

2020-09-24 Thread Sudhakar Panneerselvam
Hello Mike, > -Original Message- > From: Mike Snitzer [mailto:snit...@redhat.com] > Sent: Wednesday, September 23, 2020 7:28 PM > To: Sudhakar Panneerselvam > Cc: a...@redhat.com; dm-devel@redhat.com; dm-cr...@saout.de; > mpato...@redhat.com; damien.lem...@wdc.com; Shirley Ma > ; ssudhak.

Re: [dm-devel] [PATCH 07/13] block: lift setting the readahead size into the block layer

2020-09-24 Thread Martin K. Petersen
Christoph, > Drivers shouldn't really mess with the readahead size, as that is a VM > concept. Instead set it based on the optimal I/O size by lifting the > algorithm from the md driver when registering the disk. Also set > bdi->io_pages there as well by applying the same scheme based on > max

Re: [dm-devel] [PATCH 06/13] md: update the optimal I/O size on reshape

2020-09-24 Thread Martin K. Petersen
Christoph, > The raid5 and raid10 drivers currently update the read-ahead size, but > not the optimal I/O size on reshape. To prepare for deriving the > read-ahead size from the optimal I/O size make sure it is updated as > well. Reviewed-by: Martin K. Petersen -- Martin K. Petersen Or

Re: [dm-devel] [PATCH 03/13] bcache: inherit the optimal I/O size

2020-09-24 Thread Martin K. Petersen
Christoph, > Inherit the optimal I/O size setting just like the readahead window, > as any reason to do larger I/O does not apply to just readahead. Looks fine. Reviewed-by: Martin K. Petersen -- Martin K. Petersen Oracle Linux Engineering -- dm-devel mailing list dm-devel@redhat.com

Re: [dm-devel] [PATCH 2/3] dm: add support for passing through inline crypto support

2020-09-24 Thread Eric Biggers
On Thu, Sep 24, 2020 at 09:46:49AM -0400, Mike Snitzer wrote: > > > Can you help me better understand the expected consumer of this code? > > > If you have something _real_ please be explicit. It makes justifying > > > supporting niche code like this more tolerable. > > > > So the motivation for t

Re: [dm-devel] [PATCH 07/13] block: lift setting the readahead size into the block layer

2020-09-24 Thread Mike Snitzer
On Thu, Sep 24 2020 at 2:51am -0400, Christoph Hellwig wrote: > Drivers shouldn't really mess with the readahead size, as that is a VM > concept. Instead set it based on the optimal I/O size by lifting the > algorithm from the md driver when registering the disk. Also set > bdi->io_pages there

Re: [dm-devel] dm-table: set non-zero q->limits.discard_granularity

2020-09-24 Thread Mike Snitzer
On Thu, Sep 24 2020 at 10:34am -0400, Mikulas Patocka wrote: > Hi Mike > > Could you send this to Linus before v5.9 is released? > > Mikulas > > > > From: Mikulas Patocka > > If q->limits.discard_granularity is zero, the block core will warn in > __blkdev_issue_discard. This patch sets it

Re: [dm-devel] [PATCH 07/13] block: lift setting the readahead size into the block layer

2020-09-24 Thread Jan Kara
On Thu 24-09-20 08:51:34, Christoph Hellwig wrote: > Drivers shouldn't really mess with the readahead size, as that is a VM > concept. Instead set it based on the optimal I/O size by lifting the > algorithm from the md driver when registering the disk. Also set > bdi->io_pages there as well by ap

[dm-devel] [PATCH] dm-table: set non-zero q->limits.discard_granularity

2020-09-24 Thread Mikulas Patocka
Hi Mike Could you send this to Linus before v5.9 is released? Mikulas From: Mikulas Patocka If q->limits.discard_granularity is zero, the block core will warn in __blkdev_issue_discard. This patch sets it to a minimum - 512 bytes. Signed-off-by: Mikulas Patocka Fixes: b35fd7422c2f ("block:

Re: [dm-devel] [PATCH 2/3] dm: add support for passing through inline crypto support

2020-09-24 Thread Mike Snitzer
On Thu, Sep 24 2020 at 3:38am -0400, Satya Tangirala wrote: > On Wed, Sep 23, 2020 at 09:21:03PM -0400, Mike Snitzer wrote: > > On Wed, Sep 09 2020 at 7:44pm -0400, > > Satya Tangirala wrote: > > > > > From: Eric Biggers > > > > > > Update the device-mapper core to support exposing the inli

[dm-devel] [PATCH 10/23] multipathd: add helper for systemd notification at exit

2020-09-24 Thread mwilck
From: Martin Wilck Add sd_notify_exit(). Signed-off-by: Martin Wilck --- multipathd/main.c | 22 +- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index 44c1bd1..e742aa5 100644 --- a/multipathd/main.c +++ b/multipathd/ma

Re: [dm-devel] [PATCH 2/3] dm: add support for passing through inline crypto support

2020-09-24 Thread Mike Snitzer
On Thu, Sep 24 2020 at 3:17am -0400, Satya Tangirala wrote: > On Wed, Sep 23, 2020 at 09:14:39PM -0400, Mike Snitzer wrote: > > On Mon, Sep 21 2020 at 8:32pm -0400, > > Eric Biggers wrote: > > > > > On Wed, Sep 09, 2020 at 11:44:21PM +, Satya Tangirala wrote: > > > > From: Eric Biggers >

[dm-devel] [PATCH 04/23] multipathd: move vecs desctruction into cleanup function

2020-09-24 Thread mwilck
From: Martin Wilck This will make it easer to move the stuff around later. The only functional change is that map destuction now happens after joining all threads, which should actually improve robustness. Signed-off-by: Martin Wilck --- multipathd/main.c | 64 +

[dm-devel] [PATCH 18/23] libmultipath: fix log_thread startup and teardown

2020-09-24 Thread mwilck
From: Martin Wilck This fixes several issues with the log_thread. First, the running flag logq_running should be set by the thread itself, not by log_thread_start()/_stop(). Second, the thread was both cancelled and terminated via a flag (again, logq_running). It's sufficient, and better, to just

[dm-devel] [PATCH 14/23] libmultipath: add libmp_dm_exit()

2020-09-24 Thread mwilck
From: Martin Wilck This function prepares for calling dm_lib_exit() on program exit. It undoes changes to libdm internals done by libmultipath. It doesn't call dm_lib_exit(), as the caller may want to keep libdm active. Signed-off-by: Martin Wilck --- libmultipath/config.c| 1 + libmultip

[dm-devel] [PATCH 12/23] multipathd: unwatch_all_dmevents: check if waiter is initialized

2020-09-24 Thread mwilck
From: Martin Wilck Signed-off-by: Martin Wilck --- multipathd/dmevents.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/multipathd/dmevents.c b/multipathd/dmevents.c index b561cbf..f52f597 100644 --- a/multipathd/dmevents.c +++ b/multipathd/dmevents.c @@ -257,6 +257,8 @@ void unwatch_all

[dm-devel] [PATCH 07/23] multipathd: move conf destruction into separate function

2020-09-24 Thread mwilck
From: Martin Wilck Also removing the comment about dlog() and dm_write_log(). dlog() can cope with get_multipath_config() returning NULL, and dm_write_log() hasn't accessed the configuration for a while. Signed-off-by: Martin Wilck --- multipathd/main.c | 24 ++-- 1 file ch

[dm-devel] [PATCH 23/23] multipath-tools: mpath-tools.supp: file with valgrind suppressions

2020-09-24 Thread mwilck
From: Martin Wilck These leaks are caused by other libraries (libsystemd, glibc, libgcrypt) and should be ignored when debugging with valgrind Usage example: valgrind --suppressions=mpath-tools.supp \ --leak-check=full --show-leak-kinds=all $COMMAND Signed-off-by: Martin Wilck --- third-

[dm-devel] [PATCH 16/23] libmultipath: log_thread_stop(): check if logarea is initialized

2020-09-24 Thread mwilck
From: Martin Wilck Signed-off-by: Martin Wilck --- libmultipath/log_pthread.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/libmultipath/log_pthread.c b/libmultipath/log_pthread.c index 15baef8..0c327ff 100644 --- a/libmultipath/log_pthread.c +++ b/libmultipath/log_pthread.c @@ -112,6

[dm-devel] [PATCH 20/23] multipath: use atexit() for cleanup handlers

2020-09-24 Thread mwilck
From: Martin Wilck Signed-off-by: Martin Wilck --- multipath/main.c | 39 ++- 1 file changed, 18 insertions(+), 21 deletions(-) diff --git a/multipath/main.c b/multipath/main.c index 9ae46ed..049a36f 100644 --- a/multipath/main.c +++ b/multipath/main.c @@ -4

[dm-devel] [PATCH 11/23] multipathd: child(): call cleanups in failure case, too

2020-09-24 Thread mwilck
From: Martin Wilck So far we haven't called any cleanup code if child() failed. Fix it. Signed-off-by: Martin Wilck --- multipathd/main.c | 10 -- 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index e742aa5..3c8f893 100644 --- a/mul

[dm-devel] [PATCH 13/23] multipathd: print error message if config can't be loaded

2020-09-24 Thread mwilck
From: Martin Wilck Signed-off-by: Martin Wilck --- multipathd/main.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/multipathd/main.c b/multipathd/main.c index 3c8f893..483b3c7 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -3074,8 +3074,10 @@ child (__attribu

[dm-devel] [PATCH 19/23] multipathd: move cleanup_{prio, checkers, foreign} to libmultipath_exit

2020-09-24 Thread mwilck
From: Martin Wilck Signed-off-by: Martin Wilck --- libmpathpersist/mpath_persist.c | 2 -- libmultipath/config.c | 4 libmultipath/libmultipath.version | 5 + multipathd/main.c | 3 --- 4 files changed, 5 insertions(+), 9 deletions(-) diff --git a/libmpat

[dm-devel] [PATCH 09/23] multipathd: close pidfile on exit

2020-09-24 Thread mwilck
From: Martin Wilck It seems we've been doing this only in the failure case, for ages. Signed-off-by: Martin Wilck --- multipathd/main.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/multipathd/main.c b/multipathd/main.c index 3db051b..44c1bd1 100644 --- a/multipathd/ma

[dm-devel] [PATCH 00/23] libmultipath: improve cleanup on exit

2020-09-24 Thread mwilck
From: Martin Wilck Hi Christophe, hi Ben, hi lixiaokeng, this series was inspired by lixiaokeng's recent posting "[QUESTION] memory leak in main (multipath)". It implements my first idea, registering cleanup handlers with atexit(). However it turned out to be quite complex. In particular multipa

[dm-devel] [PATCH 05/23] multipathd: make some globals static

2020-09-24 Thread mwilck
From: Martin Wilck Signed-off-by: Martin Wilck --- multipathd/main.c | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index 2642570..1cdbff1 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -116,19 +116,19 @@ struc

[dm-devel] [PATCH 17/23] multipathd: add cleanup_child() exit handler

2020-09-24 Thread mwilck
From: Martin Wilck cleanup_child() calls all cleanups in the right order, in an exit handler. Signed-off-by: Martin Wilck --- multipathd/main.c | 48 +++ 1 file changed, 28 insertions(+), 20 deletions(-) diff --git a/multipathd/main.c b/multipathd/m

[dm-devel] [PATCH 21/23] mpathpersist: use atexit() for cleanup handlers

2020-09-24 Thread mwilck
From: Martin Wilck Signed-off-by: Martin Wilck --- mpathpersist/main.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/mpathpersist/main.c b/mpathpersist/main.c index 3c2e657..14245cc 100644 --- a/mpathpersist/main.c +++ b/mpathpersist/main.c @@ -641,11 +641,10 @@ int m

[dm-devel] [PATCH 15/23] multipathd: fixup libdm deinitialization

2020-09-24 Thread mwilck
From: Martin Wilck With libmp_dm_exit() in place, we can make sure that the calls are made in the right order. Signed-off-by: Martin Wilck --- multipathd/main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index 483b3c7..f9749e5

[dm-devel] [PATCH 01/23] multipathd: uxlsnr: avoid deadlock on exit

2020-09-24 Thread mwilck
From: Martin Wilck The uxlsnr wouldn't always release the client lock when cancelled, causing a deadlock in uxsock_cleanup(). While this hasn't been caused by commit 3d611a2, the deadlock seems to have become much more likely after that patch. Solving this means that we have to treat reallocation

[dm-devel] [PATCH 02/23] multipathd: Fix liburcu memory leak

2020-09-24 Thread mwilck
From: Martin Wilck Fix this leak in multipathd, reported by valgrind, that messes up multipathd's otherwise clean leak report: ==23823== 336 bytes in 1 blocks are possibly lost in loss record 3 of 3 ==23823==at 0x483AB65: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==2

[dm-devel] [PATCH 06/23] multipathd: move threads destruction into separate function

2020-09-24 Thread mwilck
From: Martin Wilck Also, introduce booleans that indicate a certain thread has been started successfully. Using these booleans, we can avoid crashing by cancelling threads that have never been started. Signed-off-by: Martin Wilck --- multipathd/main.c | 76 +++--

[dm-devel] [PATCH 03/23] multipathd: move handling of io_err_stat_attr into libmultipath

2020-09-24 Thread mwilck
From: Martin Wilck This thread attribute can be dynamically initialized and destroyed. No need to carry it along in multipathd. Removal of the symbol requires a new ABI version. Signed-off-by: Martin Wilck --- libmultipath/io_err_stat.c| 7 +-- libmultipath/libmultipath.version |

[dm-devel] [PATCH 22/23] multipath: fix leaks in check_path_valid()

2020-09-24 Thread mwilck
From: Martin Wilck There were two leaks in check_path_valid(): if path status was successfully determined before calling store_pathvec(), free_path() wasn't called. Also, if an error exit occured, neither cleanup function was called. This patch fixes both, at the cost of using "static" for the p

[dm-devel] [PATCH 08/23] multipathd: move pid destruction into separate function

2020-09-24 Thread mwilck
From: Martin Wilck Signed-off-by: Martin Wilck --- multipathd/main.c | 10 +++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index 4d5b40b..3db051b 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -2892,6 +2892,12 @@ set_oom_

Re: [dm-devel] [PATCH 1/2] block: add QUEUE_FLAG_NOWAIT

2020-09-24 Thread Christoph Hellwig
On Wed, Sep 23, 2020 at 04:06:51PM -0400, Mike Snitzer wrote: > Add QUEUE_FLAG_NOWAIT to allow a block device to advertise support for > REQ_NOWAIT. Bio-based devices may set QUEUE_FLAG_NOWAIT where > applicable. > > Update QUEUE_FLAG_MQ_DEFAULT to include QUEUE_FLAG_NOWAIT. Also > update submit_

Re: [dm-devel] [PATCH 2/3] dm: add support for passing through inline crypto support

2020-09-24 Thread Mike Snitzer
On Thu, Sep 24 2020 at 3:48am -0400, Satya Tangirala wrote: > On Wed, Sep 23, 2020 at 09:21:03PM -0400, Mike Snitzer wrote: > > On Wed, Sep 09 2020 at 7:44pm -0400, > > Satya Tangirala wrote: > > > > > From: Eric Biggers > > > > > > Update the device-mapper core to support exposing the inli

[dm-devel] [PATCH v2 15/21] multipath: use {get_put}_multipath_config from libmultipath

2020-09-24 Thread mwilck
From: Martin Wilck Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipath/main.c | 20 1 file changed, 4 insertions(+), 16 deletions(-) diff --git a/multipath/main.c b/multipath/main.c index dc4974b..4bbfce9 100644 --- a/multipath/main.c +++ b/multipath

[dm-devel] [PATCH v2 04/21] multipathd: send "RELOADING=1" to systemd on DAEMON_CONFIGURE state

2020-09-24 Thread mwilck
From: Martin Wilck The logic is as follows: child() sets DAEMON_IDLE status after DAEMON_CONFIGURE when reconfigure() has finished. The only other state change that can race with that is DAEMON_SHUTDOWN. Other state changes will wait for DAEMON_IDLE first (see set_config_state()). When DAEMON_CON

[dm-devel] [PATCH v2 21/21] multipathd: remove logsink and udev

2020-09-24 Thread mwilck
From: Martin Wilck We can use the symbols from libmultipath now. Signed-off-by: Martin Wilck --- multipathd/main.c | 9 +++-- 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index 00b66ba..c5c374b 100644 --- a/multipathd/main.c +++ b/mult

[dm-devel] [PATCH v2 02/21] multipathd: avoid sending "READY=1" to systemd on early shutdown

2020-09-24 Thread mwilck
From: Martin Wilck If multipathd gets a shutdown request during initial reconfigure(), it shouldn't send "READY=1" to systemd. Ensure this by sending "READY=1" via post_config_state(). Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipathd/main.c | 16 +++- 1 f

[dm-devel] [PATCH v2 00/21] multipath-tools: shutdown, libdevmapper races, globals

2020-09-24 Thread mwilck
From: Martin Wilck Hi Christophe, hi Ben, this series contains a number of patches I've wanted to do for some time. It's based on my "upstream-queue" tree (https://github.com/openSUSE/multipath-tools/tree/upstream-queue), plus my previously submitted "multipath-tools:add linker version scripts"

[dm-devel] [PATCH v2 09/21] multipath-tools: don't call dm_lib_release() any more

2020-09-24 Thread mwilck
From: Martin Wilck The purpose of dm_lib_release() is to release stacked device node operations in libdevmapper. This is functionality we don't need and use any more, as we rely on udev to set up device nodes and symlinks. We always set DM_UDEV_DISABLE_LIBRARY_FALLBACK when we run dm tasks. In t

[dm-devel] [PATCH v2 14/21] libmpathpersist: allow using libmultipath {get, put}_multipath_config

2020-09-24 Thread mwilck
From: Martin Wilck Provide an alternative init function libmpathpersist_init() which avoids allocating a new struct config, simply using libmultipath's internal implementation. Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- libmpathpersist/libmpathpersist.version | 6 l

[dm-devel] [PATCH v2 19/21] libmpathpersist: call libmultipath_{init, exit}()

2020-09-24 Thread mwilck
From: Martin Wilck Have libmpathpersist_{init,exit} do the udev initialization, too. Signed-off-by: Martin Wilck --- libmpathpersist/mpath_persist.c | 11 --- libmpathpersist/mpath_persist.h | 9 ++--- 2 files changed, 14 insertions(+), 6 deletions(-) diff --git a/libmpathpersist

[dm-devel] [PATCH v2 16/21] mpathpersist: use {get, put}_multipath_config() from libmultipath

2020-09-24 Thread mwilck
From: Martin Wilck Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- mpathpersist/main.c | 16 ++-- 1 file changed, 2 insertions(+), 14 deletions(-) diff --git a/mpathpersist/main.c b/mpathpersist/main.c index a6a3bcf..278e48f 100644 --- a/mpathpersist/main.c +++ b/m

[dm-devel] [PATCH v2 10/21] libmultipath: devmapper: refactor libdm version determination

2020-09-24 Thread mwilck
From: Martin Wilck As one step towards bundling all possibly racy libdm init calls into a single function, split the code for determining and checking versions of libdm and kernel components. Provide a generic helper libmp_get_version() that makes sure the versions are "lazily" initialized. Note

[dm-devel] [PATCH v2 06/21] multipathd: generalize and fix wait_for_state_change_if()

2020-09-24 Thread mwilck
From: Martin Wilck It's unlikely but not impossible that other threads change the state while we're waiting, and if we grab the lock again, it's still not what we wanted. We need to continue waiting until either the condition is met, or time timeout expired. Moreover, generalize this code so tha

[dm-devel] [PATCH v2 13/21] libmultipath: provide defaults for {get, put}_multipath_config

2020-09-24 Thread mwilck
From: Martin Wilck Add an implementation of get_multipath_config() and put_multipath_config() to libmultipath. The linker's symbol lookup rules will make sure that applications can override these functions if they need to. Defining these functions in libmultipath avoids the need to provide stubs

[dm-devel] [PATCH v2 07/21] multipathd: set_config_state(): avoid code duplication

2020-09-24 Thread mwilck
From: Martin Wilck Use __post_config_state() and __wait_for_state_change(). This way __post_config_state() is the only place where running_state is ever changed, and we avoid code duplication. Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipathd/main.c | 23 +

[dm-devel] [PATCH v2 17/21] libmultipath: add udev and logsink symbols

2020-09-24 Thread mwilck
From: Martin Wilck With these symbols added, applications using libmultipath don't need to define global variables "udev" and "logsink" any more. This comes at the cost of having to call an init function. Currently, libmultipath_init() does nothing but initialize "udev". The linker's symbol look

[dm-devel] [PATCH v2 18/21] multipath: remove logsink and udev

2020-09-24 Thread mwilck
From: Martin Wilck We can use libmultipath's symbols now. Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipath/main.c | 7 ++- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/multipath/main.c b/multipath/main.c index 4bbfce9..9ae46ed 100644 --- a/multipa

[dm-devel] [PATCH v2 11/21] libmultipath: protect racy libdevmapper calls with a mutex

2020-09-24 Thread mwilck
From: Martin Wilck dm_udev_wait() and dm_task_run() may access global / static state in libdevmapper. They need to be protected by a lock in in our multithreaded library. Cc: lixiaok...@huawei.com Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- libmultipath/devmapper.c

[dm-devel] [PATCH v2 01/21] multipathd: allow shutdown during configure()

2020-09-24 Thread mwilck
From: Martin Wilck reconfigure() can be a long-running operation; both initial path discovery and initial map setup can take a long time. Allow the main program to indicate that the process should be interrupted if a shutdown signal was received. We take advantage of the dynamic linker's symbol

[dm-devel] [PATCH v2 08/21] multipathd: cancel threads early during shutdown

2020-09-24 Thread mwilck
From: Martin Wilck Cancel the other threads before taking vecs->lock. This avoids delays during shutdown caused e.g. by the checker thread holding the vecs lock. Note: this makes it possible that cancelled threads leak memory, because they can now be cancelled before having released the vecs loc

[dm-devel] [PATCH v2 20/21] mpathpersist: remove logsink and udev

2020-09-24 Thread mwilck
From: Martin Wilck We can use libmultipath's internal symbols now. The libmultipath initialization is taken care of by libmpathpersist_init(). Signed-off-by: Martin Wilck --- mpathpersist/main.c | 6 -- 1 file changed, 6 deletions(-) diff --git a/mpathpersist/main.c b/mpathpersist/main.c

[dm-devel] [PATCH v2 05/21] multipathd: use volatile qualifier for running_state

2020-09-24 Thread mwilck
From: Martin Wilck While we access running_state only under the config_lock, we sometimes do in a loop. Use the volatile qualifier to make sure compilers can't optimize away the loads. Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipathd/main.c | 2 +- 1 file changed, 1

[dm-devel] [PATCH v2 03/21] multipathd: send "STOPPING=1" to systemd on shutdown

2020-09-24 Thread mwilck
From: Martin Wilck Inform systemd that the daemon is shutting down. See sd_notify(3). Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipathd/main.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/multipathd/main.c b/multipathd/main.c index 53a22a4..c

[dm-devel] [PATCH v2 12/21] libmultipath: constify file argument in config parser

2020-09-24 Thread mwilck
From: Martin Wilck Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- libmultipath/config.c | 3 +-- libmultipath/config.h | 2 +- libmultipath/parser.c | 9 + libmultipath/parser.h | 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/libmultipath/config.

[dm-devel] [PATCH 08/11] libmultipath: create separate .so for unit tests

2020-09-24 Thread mwilck
From: Martin Wilck The unit tests use a lot of internal symbols that we don't want to add to the ABI if we don't have to. As long as we don't have to make incompatible changes to functions, we can work around that by simply using a non-versioned library for the unit tests. Therefore we add a sepe

[dm-devel] [PATCH 00/11] multipath-tools: add linker version scripts

2020-09-24 Thread mwilck
From: Martin Wilck Hi Christophe, hi Ben, Patch 1-5 are small fixes, the first two resent from an earlier submission. Patch 6ff. add version scripts for the linker to libmultipath, libmpathpersist, and libmpathcmd. Is it useful to do this for libmultipath? We have always said that this is not a

  1   2   >