[PATCH 4.4 24/26] blk: improve order of bio handling in generic_make_request()

2017-04-06 Thread Greg Kroah-Hartman
4.4-stable review patch. If anyone has any objections, please let me know. -- From: NeilBrown commit 79bd99596b7305ab08109a8bf44a6a4511dbf1cd upstream. To avoid recursion on the kernel stack when stacked block devices are in use, generic_make_request() will, when called recurs

[PATCH 4.9 68/72] blk: improve order of bio handling in generic_make_request()

2017-04-06 Thread Greg Kroah-Hartman
4.9-stable review patch. If anyone has any objections, please let me know. -- From: NeilBrown commit 79bd99596b7305ab08109a8bf44a6a4511dbf1cd upstream. To avoid recursion on the kernel stack when stacked block devices are in use, generic_make_request() will, when called recurs

[PATCH 4.10 74/81] blk: improve order of bio handling in generic_make_request()

2017-04-06 Thread Greg Kroah-Hartman
4.10-stable review patch. If anyone has any objections, please let me know. -- From: NeilBrown commit 79bd99596b7305ab08109a8bf44a6a4511dbf1cd upstream. To avoid recursion on the kernel stack when stacked block devices are in use, generic_make_request() will, when called recur

Re: [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-10 Thread NeilBrown
On Fri, Mar 10 2017, Lars Ellenberg wrote: >> --- a/block/blk-core.c >> +++ b/block/blk-core.c >> @@ -1975,7 +1975,14 @@ generic_make_request_checks(struct bio *bio) >> */ >> blk_qc_t generic_make_request(struct bio *bio) >> { >> - struct bio_list bio_list_on_stack; >> + /* >> +

Re: [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-10 Thread Lars Ellenberg
On Fri, Mar 10, 2017 at 04:07:58PM +0100, Jack Wang wrote: > On 10.03.2017 15:55, Mikulas Patocka wrote: > > On Fri, 10 Mar 2017, Mike Snitzer wrote: > >> On Fri, Mar 10 2017 at 7:34am -0500, > >> Lars Ellenberg wrote: > >> > --- a/block/blk-core.c > +++ b/block/blk-core.c > @@ -19

Re: [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-10 Thread Mike Snitzer
On Fri, Mar 10 2017 at 10:07am -0500, Jack Wang wrote: > > > On 10.03.2017 15:55, Mikulas Patocka wrote: > > > > > > On Fri, 10 Mar 2017, Mike Snitzer wrote: > > > >> On Fri, Mar 10 2017 at 7:34am -0500, > >> Lars Ellenberg wrote: > >> > --- a/block/blk-core.c > +++ b/block/blk-c

Re: [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-10 Thread Jack Wang
On 10.03.2017 15:55, Mikulas Patocka wrote: > > > On Fri, 10 Mar 2017, Mike Snitzer wrote: > >> On Fri, Mar 10 2017 at 7:34am -0500, >> Lars Ellenberg wrote: >> --- a/block/blk-core.c +++ b/block/blk-core.c @@ -1975,7 +1975,14 @@ generic_make_request_checks(struct bio *bio) >>

Re: [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-10 Thread Mikulas Patocka
On Fri, 10 Mar 2017, Mike Snitzer wrote: > On Fri, Mar 10 2017 at 7:34am -0500, > Lars Ellenberg wrote: > > > > --- a/block/blk-core.c > > > +++ b/block/blk-core.c > > > @@ -1975,7 +1975,14 @@ generic_make_request_checks(struct bio *bio) > > > */ > > > blk_qc_t generic_make_request(struct

Re: [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-10 Thread Mike Snitzer
On Fri, Mar 10 2017 at 7:34am -0500, Lars Ellenberg wrote: > > --- a/block/blk-core.c > > +++ b/block/blk-core.c > > @@ -1975,7 +1975,14 @@ generic_make_request_checks(struct bio *bio) > > */ > > blk_qc_t generic_make_request(struct bio *bio) > > { > > - struct bio_list bio_list_on_sta

Re: [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-10 Thread Lars Ellenberg
> --- a/block/blk-core.c > +++ b/block/blk-core.c > @@ -1975,7 +1975,14 @@ generic_make_request_checks(struct bio *bio) > */ > blk_qc_t generic_make_request(struct bio *bio) > { > - struct bio_list bio_list_on_stack; > + /* > +* bio_list_on_stack[0] contains bios submitted b

Re: [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-09 Thread NeilBrown
On Thu, Mar 09 2017, Jens Axboe wrote: > On 03/09/2017 09:32 PM, NeilBrown wrote: >> >> I started looking further at the improvements we can make once >> generic_make_request is fixed, and realised that I had missed an >> important detail in this patch. >> Several places test current->bio_list, a

Re: [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-09 Thread Jens Axboe
On 03/09/2017 09:38 PM, Jens Axboe wrote: > On 03/09/2017 09:32 PM, NeilBrown wrote: >> >> I started looking further at the improvements we can make once >> generic_make_request is fixed, and realised that I had missed an >> important detail in this patch. >> Several places test current->bio_list,

Re: [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-09 Thread Jens Axboe
On 03/09/2017 09:32 PM, NeilBrown wrote: > > I started looking further at the improvements we can make once > generic_make_request is fixed, and realised that I had missed an > important detail in this patch. > Several places test current->bio_list, and two actually edit the list. > With this chan

[PATCH 1/5 v3] blk: improve order of bio handling in generic_make_request()

2017-03-09 Thread NeilBrown
To avoid recursion on the kernel stack when stacked block devices are in use, generic_make_request() will, when called recursively, queue new requests for later handling. They will be handled when the make_request_fn for the current bio completes. If any bios are submitted by a make_request_fn,

Re: [PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-09 Thread NeilBrown
it from some easy-to-fix places. The series summary is below. NeilBrown NeilBrown (5): blk: improve order of bio handling in generic_make_request() blk: remove bio_set arg from blk_queue_split() blk: make the bioset rescue_workqueue optional. blk: use non-rescuing bios

Re: blk: improve order of bio handling in generic_make_request()

2017-03-08 Thread NeilBrown
On Wed, Mar 08 2017, Mikulas Patocka wrote: > On Wed, 8 Mar 2017, NeilBrown wrote: >> >> I don't think this will fix the DM snapshot deadlock by itself. >> Rather, it make it possible for some internal changes to DM to fix it. >> The DM change might be something vaguely like: >> >> diff --git a/

Re: blk: improve order of bio handling in generic_make_request()

2017-03-08 Thread Lars Ellenberg
On 8 March 2017 at 17:40, Mikulas Patocka wrote: > > On Wed, 8 Mar 2017, NeilBrown wrote: > > I don't think this will fix the DM snapshot deadlock by itself. > > Rather, it make it possible for some internal changes to DM to fix it. > > The DM change might be something vaguely like: > > > > diff -

Re: blk: improve order of bio handling in generic_make_request()

2017-03-08 Thread Mikulas Patocka
On Wed, 8 Mar 2017, NeilBrown wrote: > On Tue, Mar 07 2017, Mike Snitzer wrote: > > > On Tue, Mar 07 2017 at 12:05pm -0500, > > Jens Axboe wrote: > > > >> On 03/07/2017 09:52 AM, Mike Snitzer wrote: > >> > On Tue, Mar 07 2017 at 3:49am -0500, > >> > Jack Wang wrote: > >> > > >> >> > >> >> >

Re: blk: improve order of bio handling in generic_make_request()

2017-03-08 Thread Lars Ellenberg
On 7 March 2017 at 17:52, Mike Snitzer wrote: > > On 06.03.2017 21:18, Jens Axboe wrote: > > > I like the change, and thanks for tackling this. It's been a pending > > > issue for way too long. I do think we should squash Jack's patch > > > into the original, as it does clean up the code nicely.

Re: blk: improve order of bio handling in generic_make_request()

2017-03-07 Thread Mike Snitzer
On Tue, Mar 07 2017 at 3:29pm -0500, NeilBrown wrote: > On Tue, Mar 07 2017, Mike Snitzer wrote: > > > On Tue, Mar 07 2017 at 12:05pm -0500, > > Jens Axboe wrote: > > > >> On 03/07/2017 09:52 AM, Mike Snitzer wrote: > >> > > >> > In addition to Jack's MD raid test there is a DM snapshot deadl

Re: blk: improve order of bio handling in generic_make_request()

2017-03-07 Thread NeilBrown
On Tue, Mar 07 2017, Mike Snitzer wrote: > On Tue, Mar 07 2017 at 12:05pm -0500, > Jens Axboe wrote: > >> On 03/07/2017 09:52 AM, Mike Snitzer wrote: >> > On Tue, Mar 07 2017 at 3:49am -0500, >> > Jack Wang wrote: >> > >> >> >> >> >> >> On 06.03.2017 21:18, Jens Axboe wrote: >> >>> On 03/05/20

[PATCH v2] blk: improve order of bio handling in generic_make_request()

2017-03-07 Thread NeilBrown
To avoid recursion on the kernel stack when stacked block devices are in use, generic_make_request() will, when called recursively, queue new requests for later handling. They will be handled when the make_request_fn for the current bio completes. If any bios are submitted by a make_request_fn,

Re: [PATCH] blk: improve order of bio handling in generic_make_request()

2017-03-07 Thread Jack Wang
On 07.03.2017 16:46, Pavel Machek wrote: > On Mon 2017-03-06 10:43:59, Jack Wang wrote: >> >> >> On 06.03.2017 05:40, NeilBrown wrote: >>> On Fri, Mar 03 2017, Jack Wang wrote: Thanks Neil for pushing the fix. We can optimize generic_make_request a little bit: - assign bi

Re: blk: improve order of bio handling in generic_make_request()

2017-03-07 Thread Mike Snitzer
On Tue, Mar 07 2017 at 3:49am -0500, Jack Wang wrote: > > > On 06.03.2017 21:18, Jens Axboe wrote: > > On 03/05/2017 09:40 PM, NeilBrown wrote: > >> On Fri, Mar 03 2017, Jack Wang wrote: > >>> > >>> Thanks Neil for pushing the fix. > >>> > >>> We can optimize generic_make_request a little bit:

Re: blk: improve order of bio handling in generic_make_request()

2017-03-07 Thread Mike Snitzer
On Tue, Mar 07 2017 at 12:05pm -0500, Jens Axboe wrote: > On 03/07/2017 09:52 AM, Mike Snitzer wrote: > > On Tue, Mar 07 2017 at 3:49am -0500, > > Jack Wang wrote: > > > >> > >> > >> On 06.03.2017 21:18, Jens Axboe wrote: > >>> On 03/05/2017 09:40 PM, NeilBrown wrote: > On Fri, Mar 03 201

Re: blk: improve order of bio handling in generic_make_request()

2017-03-07 Thread Jens Axboe
On 03/07/2017 09:52 AM, Mike Snitzer wrote: > On Tue, Mar 07 2017 at 3:49am -0500, > Jack Wang wrote: > >> >> >> On 06.03.2017 21:18, Jens Axboe wrote: >>> On 03/05/2017 09:40 PM, NeilBrown wrote: On Fri, Mar 03 2017, Jack Wang wrote: > > Thanks Neil for pushing the fix. > >

Re: [PATCH] blk: improve order of bio handling in generic_make_request()

2017-03-07 Thread Jens Axboe
On 03/07/2017 08:46 AM, Pavel Machek wrote: > On Mon 2017-03-06 10:43:59, Jack Wang wrote: >> >> >> On 06.03.2017 05:40, NeilBrown wrote: >>> On Fri, Mar 03 2017, Jack Wang wrote: Thanks Neil for pushing the fix. We can optimize generic_make_request a little bit: - assign b

Re: [PATCH] blk: improve order of bio handling in generic_make_request()

2017-03-07 Thread Pavel Machek
On Mon 2017-03-06 10:43:59, Jack Wang wrote: > > > On 06.03.2017 05:40, NeilBrown wrote: > > On Fri, Mar 03 2017, Jack Wang wrote: > >> > >> Thanks Neil for pushing the fix. > >> > >> We can optimize generic_make_request a little bit: > >> - assign bio_list struct hold directly instead init and m

Re: [PATCH] blk: improve order of bio handling in generic_make_request()

2017-03-07 Thread Jack Wang
On 06.03.2017 21:18, Jens Axboe wrote: > On 03/05/2017 09:40 PM, NeilBrown wrote: >> On Fri, Mar 03 2017, Jack Wang wrote: >>> >>> Thanks Neil for pushing the fix. >>> >>> We can optimize generic_make_request a little bit: >>> - assign bio_list struct hold directly instead init and merge >>> - re

Re: [PATCH] blk: improve order of bio handling in generic_make_request()

2017-03-06 Thread Jens Axboe
On 03/05/2017 09:40 PM, NeilBrown wrote: > On Fri, Mar 03 2017, Jack Wang wrote: >> >> Thanks Neil for pushing the fix. >> >> We can optimize generic_make_request a little bit: >> - assign bio_list struct hold directly instead init and merge >> - remove duplicate code >> >> I think better to squash

Re: [PATCH] blk: improve order of bio handling in generic_make_request()

2017-03-06 Thread Jack Wang
On 06.03.2017 05:40, NeilBrown wrote: > On Fri, Mar 03 2017, Jack Wang wrote: >> >> Thanks Neil for pushing the fix. >> >> We can optimize generic_make_request a little bit: >> - assign bio_list struct hold directly instead init and merge >> - remove duplicate code >> >> I think better to squash

Re: [PATCH] blk: improve order of bio handling in generic_make_request()

2017-03-05 Thread NeilBrown
On Fri, Mar 03 2017, Jack Wang wrote: > > Thanks Neil for pushing the fix. > > We can optimize generic_make_request a little bit: > - assign bio_list struct hold directly instead init and merge > - remove duplicate code > > I think better to squash into your fix. Hi Jack, I don't object to your c

Re: [PATCH] blk: improve order of bio handling in generic_make_request()

2017-03-03 Thread Jack Wang
On 03.03.2017 06:14, NeilBrown wrote: > > [ Hi Jens, > you might have seen assorted email threads recently about > deadlocks, particular in dm-snap or md/raid1/10. Also about > the excess of rescuer threads. > I think a big part of the problem is my ancient improvement > to generic_ma

[PATCH] blk: improve order of bio handling in generic_make_request()

2017-03-02 Thread NeilBrown
[ Hi Jens, you might have seen assorted email threads recently about deadlocks, particular in dm-snap or md/raid1/10. Also about the excess of rescuer threads. I think a big part of the problem is my ancient improvement to generic_make_request to queue bios and handle them in a strict