Re: [dm-devel] [git pull] device mapper fixes for 5.3-rc6

2019-08-23 Thread pr-tracker-bot
The pull request you sent on Fri, 23 Aug 2019 11:41:53 -0400: > git://git.kernel.org/pub/scm/linux/kernel/git/device-mapper/linux-dm.git > tags/for-5.3/dm-fixes-2 has been merged into torvalds/linux.git: https://git.kernel.org/torvalds/c/dd469a456047af5eb1ee0bcfc8fe61f5940ef0e0 Thank you! --

[dm-devel] [PATCH v2 03/14] tests: add path grouping policy unit tests.

2019-08-23 Thread Benjamin Marzinski
In preparation for changing the path grouping code, add some unit tests to verify that it works correctly. The only test that currently fails (and so it being skipped) is using MULTIBUS when mp->paths is empty. All the other path grouping policies free mp->paths, even if it is empty. one_group() sh

[dm-devel] [PATCH v2 06/14] libmultipath: fix double free in pgpolicyfn error paths

2019-08-23 Thread Benjamin Marzinski
In the pgpolicy functions, if an error is encountered after alloc_pathgroup() is called, but before the path group is added to a multipath device with add_pathgroup(), the pathgroup needs to be cleaned up by calling free_pathgroup(). However, after the pathgroup has been added to the multipath devi

[dm-devel] [PATCH v2 09/14] libmultipath: make group_paths handle marginal paths

2019-08-23 Thread Benjamin Marzinski
group_paths() will now create seperate path groups for marginal and normal paths, and place all of the marginal path groups after the normal ones, in order by priority. Signed-off-by: Benjamin Marzinski --- libmultipath/pgpolicies.c | 83 +- libmultipath/swit

[dm-devel] [PATCH v2 12/14] libmutipath: deprecate delay_*_checks

2019-08-23 Thread Benjamin Marzinski
The delay_checks shaky paths detection method works the same way as the san_path_err method, but not as well, with less configurability, and with the code spread all over check_path(). The only real difference is that marks the path as marginal for a certain number of path checks instead of for a s

[dm-devel] [PATCH v2 11/14] libmultipath: add marginal_pathgroups config option

2019-08-23 Thread Benjamin Marzinski
group_paths now gets passed this to determine whether to enable marginal pathgroups. The unit tests have also been updated. Signed-off-by: Benjamin Marzinski --- libmultipath/config.h | 1 + libmultipath/configure.c | 5 +- libmultipath/dict.c | 3 + libmultipath/pgpolicies.c |

[dm-devel] [PATCH v2 05/14] tests: update pgpolicy tests to work with group_paths()

2019-08-23 Thread Benjamin Marzinski
The pgpolicy unit tests now work again, using group_paths(). test_one_group0(), which was skipped with the old path grouping code because it failed, is now working correctly. Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- tests/pgpolicy.c | 125 +++--

[dm-devel] [PATCH v2 04/14] libmultipath: add wrapper function around pgpolicyfn

2019-08-23 Thread Benjamin Marzinski
group_paths() is a wrapper around the pgpolicy functions, that pulls out the common code from the beginning and the end. For this to work, one_group() needs to change how it sets up the pathgroups vector to work like the other pgpolicy functions. This does means that the pathgroups in group_by_prio

[dm-devel] [PATCH v2 01/14] libmultipath: make vector_foreach_slot_backwards work as expected

2019-08-23 Thread Benjamin Marzinski
All of the code that uses vector_foreach_slot_backwards() treats "i" as the index of the entry "p", but the way it was coded, that wasn't the case. "i" was the number of the entry counting from 1, not 0. Reviewed-by: Martin Wilck Signed-off-by: Benjamin Marzinski --- libmultipath/vector.h | 2 +

[dm-devel] [PATCH v2 00/14] multipath marginal pathgroups

2019-08-23 Thread Benjamin Marzinski
The first patch in this series is simply a resend of my previous patch to fix vector_foreach_slot_backwards(). The rest of the patches are related to adding support for an alternative way to deal with marginal paths. Instead of delaying their reinstatement, which can cause problems if multipath isn

[dm-devel] [PATCH v2 02/14] libmultipath: add marginal paths and groups infrastructure

2019-08-23 Thread Benjamin Marzinski
This commit adds a marginal variable ot the paths and pathgroups structs. The marginal paths variable can be set by multipathd path setmarginal and cleared by multipathd path unsetmarginal All of the marginal paths on a multipath device can be cleared by multipathd map unsetmarginal Curren

[dm-devel] [PATCH v2 14/14] multipath: update man pages

2019-08-23 Thread Benjamin Marzinski
Add documentation for the marginal_pathgroups option and the (un)setmarginal commands. Signed-off-by: Benjamin Marzinski --- multipath/multipath.conf.5 | 34 ++ multipathd/multipathd.8| 19 +++ 2 files changed, 49 insertions(+), 4 deletions(-)

[dm-devel] [PATCH v2 10/14] tests: add tests for grouping marginal paths.

2019-08-23 Thread Benjamin Marzinski
Signed-off-by: Benjamin Marzinski --- tests/pgpolicy.c | 337 +-- 1 file changed, 295 insertions(+), 42 deletions(-) diff --git a/tests/pgpolicy.c b/tests/pgpolicy.c index 04a77c4c..ab09f91c 100644 --- a/tests/pgpolicy.c +++ b/tests/pgpolicy.c @@ -40,6

[dm-devel] [PATCH v2 08/14] libmultipath: make pgpolicyfn take a paths vector

2019-08-23 Thread Benjamin Marzinski
To enable future changes, mp->pgpolicyfn() now takes a vector of paths instead of always using mp->paths. Signed-off-by: Benjamin Marzinski --- libmultipath/pgpolicies.c | 38 +++--- libmultipath/pgpolicies.h | 10 +- libmultipath/structs.h| 2 +- 3 f

[dm-devel] [PATCH v2 13/14] multipathd: use marginal_pathgroups

2019-08-23 Thread Benjamin Marzinski
This commit makes the marginal_pathgroups option work with the existing methods for determining marginal paths. It also merges the code for the marginal_path and sand_path_err methods. This has the side effect of making the marginal_path code set a marginal path's state to "delayed" instead of "sh

[dm-devel] [PATCH v2 07/14] libmultipath: consolidate group_by_* functions

2019-08-23 Thread Benjamin Marzinski
group_by_node_name() and group_by_serial() are exactly the same except for how the paths are compared. group_by_prio() is different but its pathvec solves the same issue as the bitmap from the other two functions, and since we are always running sort_pathgroups() after calling pgpriorityfn, there i

[dm-devel] [git pull] device mapper fixes for 5.3-rc6

2019-08-23 Thread Mike Snitzer
Hi Linus, More fixes than usual from DM at this stage in a release but... The following changes since commit d45331b00ddb179e291766617259261c112db872: Linux 5.3-rc4 (2019-08-11 13:26:41 -0700) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/device-mapp

[dm-devel] [PATCH 2/2] dm: make dm_table_find_target return NULL

2019-08-23 Thread Mikulas Patocka
Currently, if we pass too high sector number to dm_table_find_target, it returns zeroed dm_target structure and callers test if the structure is zeroed with the macro dm_target_is_valid. However, returning NULL is common practice to indicate errors. This patch refactors the dm code, so that dm_ta

[dm-devel] [PATCH 1/2] dm table: fix invalid memory accesses with too high sector number

2019-08-23 Thread Mikulas Patocka
If the sector number is too high, dm_table_find_target should return a pointer to a zeroed dm_target structure (the caller should test it with dm_target_is_valid). However, for some table sizes, the code in dm_table_find_target that performs btree lookup will access out of bound memory structures.

Re: [dm-devel] [PATCH 6/9] dm crypt: support diskcipher

2019-08-23 Thread boojin.kim
On Wed, Aug 21, 2019 at 17:19:41PM +0900, Herbert Xu wrote: > On Wed, Aug 21, 2019 at 04:57:41PM +0900, boojin.kim wrote: > > > > Can you tell me which patch you mentioned? Is this? > > https://patches.linaro.org/project/linux-crypto/list/?series=22762 > > > > Yes this is the one. > > Cheers, I l

Re: [dm-devel] [PATCH 5/9] block: support diskcipher

2019-08-23 Thread Satya Tangirala
On Wed, Aug 21, 2019 at 5:10 AM Jens Axboe wrote: > > On 8/21/19 12:42 AM, boojin.kim wrote: > > This patch supports crypto information to be maintained via BIO > > and passed to the storage driver. > > > > To do this, 'bi_aux_private', 'REQ_CYPTE' and 'bi_dun' are added > > to the block layer. >

Re: [dm-devel] [PATCH 6/9] dm crypt: support diskcipher

2019-08-23 Thread boojin.kim
On Fri, Aug 23, 2019 at 01:28:37PM +0900, Herbert Xu wrote: > > No. If you're after total offload then the crypto API is not for > you. What we can support is the offloading of encryption/decryption > over many sectors. > > Cheers, FMP doesn't use encrypt/decrypt of crypto API because it doesn't

Re: [dm-devel] [PATCH 1/9] crypt: Add diskcipher

2019-08-23 Thread boojin.kim
On Wed, 22 Aug 2019 at 17:37, Krzysztof Kozlowski wrote: > > Your patch looks corrupted - wrapped by mailer. The easiest way > usually is to use git format-patch and git send-email - then you do > not have to worry about formatting etc. > > Best regards, > Krzysztof I'm using outlook instead of '

Re: [dm-devel] [PATCH v2] md raid0/linear: Fail BIOs if their underlying block device is gone

2019-08-23 Thread Guilherme G. Piccoli
V3 just sent: lore.kernel.org/linux-block/20190822161318.26236-1-gpicc...@canonical.com The code was merged with the 'broken' state patch. Cheers, Guilherme -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel

[dm-devel] [PATCH v3 1/2] md raid0/linear: Mark array as 'broken' and fail BIOs if a member is gone

2019-08-23 Thread Guilherme G. Piccoli
Currently md raid0/linear are not provided with any mechanism to validate if an array member got removed or failed. The driver keeps sending BIOs regardless of the state of array members, and kernel shows state 'clean' in the 'array_state' sysfs attribute. This leads to the following situation: if

Re: [dm-devel] [PATCH v3 2/2] mdadm: Introduce new array state 'broken' for raid0/linear

2019-08-23 Thread Song Liu
> On Aug 22, 2019, at 9:13 AM, Guilherme G. Piccoli > wrote: > > Currently if a md raid0/linear array gets one or more members removed while > being mounted, kernel keeps showing state 'clean' in the 'array_state' > sysfs attribute. Despite udev signaling the member device is gone, 'mdadm' >

[dm-devel] [PATCH v3 2/2] mdadm: Introduce new array state 'broken' for raid0/linear

2019-08-23 Thread Guilherme G. Piccoli
Currently if a md raid0/linear array gets one or more members removed while being mounted, kernel keeps showing state 'clean' in the 'array_state' sysfs attribute. Despite udev signaling the member device is gone, 'mdadm' cannot issue the STOP_ARRAY ioctl successfully, given the array is mounted.

Re: [dm-devel] [PATCH v2 1/2] md raid0/linear: Introduce new array state 'broken'

2019-08-23 Thread Guilherme G. Piccoli
On 21/08/2019 16:22, Song Liu wrote: > [...] > > I think this makes sense. Please send the patch and we can discuss > further while looking at the code. Thanks Song, just sent the V3: lore.kernel.org/linux-block/20190822161318.26236-1-gpicc...@canonical.com Cheers, Guilherme -- dm-devel mai

Re: [dm-devel] [PATCH v2 1/2] md raid0/linear: Introduce new array state 'broken'

2019-08-23 Thread Guilherme G. Piccoli
On 22/08/2019 05:49, Guoqing Jiang wrote: > Hi, Hi Guoqing, thanks for the review! > > On 8/16/19 3:40 PM, Guilherme G. Piccoli wrote: >> +static bool linear_is_missing_dev(struct mddev *mddev) >> +{ >> +    struct md_rdev *rdev; >> +    static int already_missing; >> +    int def_disks, work_d

Re: [dm-devel] [PATCH v3 1/2] md raid0/linear: Mark array as 'broken' and fail BIOs if a member is gone

2019-08-23 Thread Song Liu
> On Aug 22, 2019, at 9:13 AM, Guilherme G. Piccoli > wrote: > > Currently md raid0/linear are not provided with any mechanism to validate > if an array member got removed or failed. The driver keeps sending BIOs > regardless of the state of array members, and kernel shows state 'clean' > in

Re: [dm-devel] [PATCH] dm table: fix a potential array out of bounds

2019-08-23 Thread Mikulas Patocka
Hi I tested it and the bug is real - with some table sizes, dm_table_find_target will access memory out of bounds if the sector argument is beyond limit. Your patch fixes some of these cases, but not all of them. I used this script to test all possible table sizes: #!/bin/bash -e sync dmsetup