Re: nbd, nbdkit, loopback mounts and memory management

2019-02-16 Thread Wouter Verhelst
re's not enough memory for nbdkit (or nbd-client or > > > nbd-server), you might get a deadlock. > > > > Thanks for the kind words about the talk. I've added Wouter Verhelst > > & the NBD mailing list to CC. Although I did the talk because the > > sub

Re: [Nbd] [PATCH 3/4] treewide: convert PF_MEMALLOC manipulations to new helpers

2017-04-05 Thread Wouter Verhelst
On Wed, Apr 05, 2017 at 01:30:31PM +0200, Michal Hocko wrote: > On Wed 05-04-17 09:46:59, Vlastimil Babka wrote: > > We now have memalloc_noreclaim_{save,restore} helpers for robust setting and > > clearing of PF_MEMALLOC. Let's convert the code which was using the generic > > tsk_restore_flags().

Re: [PATCH][V4] nbd: add multi-connection support

2016-11-22 Thread Wouter Verhelst
Hi Josef, [cc to nbd-general added] On Thu, Nov 17, 2016 at 03:27:30PM -0500, Josef Bacik wrote: > NBD can become contended on its single connection. We have to serialize all > writes and we can only process one read response at a time. Fix this by > allowing userspace to provide multiple conne

Re: [Nbd] [PATCH][V3] nbd: add multi-connection support

2016-10-06 Thread Wouter Verhelst
On Thu, Oct 06, 2016 at 06:16:30AM -0700, Christoph Hellwig wrote: > On Thu, Oct 06, 2016 at 03:09:49PM +0200, Wouter Verhelst wrote: > > Okay, I've updated the proto.md file then, to clarify that in the case > > of multiple connections, a client MUST NOT send a flush request u

Re: [Nbd] [PATCH][V3] nbd: add multi-connection support

2016-10-06 Thread Wouter Verhelst
On Thu, Oct 06, 2016 at 03:31:55AM -0700, Christoph Hellwig wrote: > No, they match the cache flush semantics in every other storage protocol > known to me, and they match the expectations of both the Linux kernel > and any other OS or comsumer I know about perfectly. Okay, I've updated the proto.

Re: [Nbd] [PATCH][V3] nbd: add multi-connection support

2016-10-06 Thread Wouter Verhelst
On Thu, Oct 06, 2016 at 10:41:36AM +0100, Alex Bligh wrote: > Wouter, [...] > > Given that, given the issue in the previous > > paragraph, and given the uncertainty introduced with multiple > > connections, I think it is reasonable to say that a client should just > > not assume a flush touches any

Re: [Nbd] [PATCH][V3] nbd: add multi-connection support

2016-10-06 Thread Wouter Verhelst
Hi Alex, On Tue, Oct 04, 2016 at 10:35:03AM +0100, Alex Bligh wrote: > Wouter, > > I see now that it should be closer > > to the former; a more useful definition is probably something along the > > following lines: > > > >All write commands (that includes NBD_CMD_WRITE and NBD_CMD_TRIM) > >

Re: [Nbd] [PATCH][V3] nbd: add multi-connection support

2016-10-03 Thread Wouter Verhelst
Alex, Christoph, On Mon, Oct 03, 2016 at 12:34:33PM +0100, Alex Bligh wrote: > On 3 Oct 2016, at 08:57, Christoph Hellwig wrote: > >> Can you clarify what you mean by that? Why is it an "odd flush > >> definition", and how would you "properly" define it? > > > > E.g. take the defintion from NVMe

Re: [Nbd] [PATCH][V3] nbd: add multi-connection support

2016-10-03 Thread Wouter Verhelst
On Mon, Oct 03, 2016 at 12:20:49AM -0700, Christoph Hellwig wrote: > On Mon, Oct 03, 2016 at 01:47:06AM +, Josef Bacik wrote: > > It's not "broken", it's working as designed, and any fs on top of this > > patch will be perfectly safe because they all wait for their io to complete > > before iss

Re: [Nbd] [PATCH][V3] nbd: add multi-connection support

2016-10-03 Thread Wouter Verhelst
On Sun, Oct 02, 2016 at 05:17:14PM +0100, Alex Bligh wrote: > On 29 Sep 2016, at 17:59, Josef Bacik wrote: > > Huh I missed that. Yeah that's not possible for us for sure, I think my > > option > > idea is the less awful way forward if we want to address that limitation. > > Thanks, > > I th

Re: [Nbd] [PATCH][V3] nbd: add multi-connection support

2016-09-29 Thread Wouter Verhelst
On Thu, Sep 29, 2016 at 10:03:50AM -0400, Josef Bacik wrote: > So think of it like normal disks with multiple channels. We don't send > flushes > down all the hwq's to make sure they are clear, we leave that decision up to > the > application (usually a FS of course). Well, when I asked earli

Re: [Nbd] [PATCH][V3] nbd: add multi-connection support

2016-09-29 Thread Wouter Verhelst
Hi Josef, On Wed, Sep 28, 2016 at 04:01:32PM -0400, Josef Bacik wrote: > NBD can become contended on its single connection. We have to serialize all > writes and we can only process one read response at a time. Fix this by > allowing userspace to provide multiple connections to a single nbd devi

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

2016-09-15 Thread Wouter Verhelst
On Thu, Sep 15, 2016 at 05:08:21PM +0100, Alex Bligh wrote: > Wouter, > > > The server can always refuse to allow multiple connections. > > Sure, but it would be neater to warn the client of that at negotiation > stage (it would only be one flag, e.g. 'multiple connections > unsafe'). I suppose

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

2016-09-15 Thread Wouter Verhelst
On Thu, Sep 15, 2016 at 01:44:29PM +0100, Alex Bligh wrote: > > > On 15 Sep 2016, at 13:41, Christoph Hellwig wrote: > > > > On Thu, Sep 15, 2016 at 01:39:11PM +0100, Alex Bligh wrote: > >> That's probably right in the case of file-based back ends that > >> are running on a Linux OS. But gonbdse

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

2016-09-15 Thread Wouter Verhelst
On Thu, Sep 15, 2016 at 05:20:08AM -0700, Christoph Hellwig wrote: > On Thu, Sep 15, 2016 at 02:01:59PM +0200, Wouter Verhelst wrote: > > Yes. There was some discussion on that part, and we decided that setting > > the flag doesn't hurt, but the spec also clarifies that using

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

2016-09-15 Thread Wouter Verhelst
On Thu, Sep 15, 2016 at 05:01:25AM -0700, Christoph Hellwig wrote: > On Thu, Sep 15, 2016 at 01:55:14PM +0200, Wouter Verhelst wrote: > > If that's not a write barrier, then I was using the wrong terminology > > (and offer my apologies for the confusion). > > It'

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

2016-09-15 Thread Wouter Verhelst
On Thu, Sep 15, 2016 at 04:52:17AM -0700, Christoph Hellwig wrote: > On Thu, Sep 15, 2016 at 12:46:07PM +0100, Alex Bligh wrote: > > Essentially NBD does supports FLUSH/FUA like this: > > > > https://www.kernel.org/doc/Documentation/block/writeback_cache_control.txt > > > > IE supports the same F

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

2016-09-15 Thread Wouter Verhelst
On Thu, Sep 15, 2016 at 04:38:07AM -0700, Christoph Hellwig wrote: > On Thu, Sep 15, 2016 at 12:49:35PM +0200, Wouter Verhelst wrote: > > A while back, we spent quite some time defining the semantics of the > > various commands in the face of the NBD_CMD_FLUSH and NBD_CMD_FLA

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

2016-09-15 Thread Wouter Verhelst
On Thu, Sep 15, 2016 at 12:09:28PM +0100, Alex Bligh wrote: > Wouter, Josef, (& Eric) > > > On 15 Sep 2016, at 11:49, Wouter Verhelst wrote: > > > > Hi, > > > > On Fri, Sep 09, 2016 at 10:02:03PM +0200, Wouter Verhelst wrote: > >> I see some pra

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

2016-09-15 Thread Wouter Verhelst
Hi, On Fri, Sep 09, 2016 at 10:02:03PM +0200, Wouter Verhelst wrote: > I see some practical problems with this: [...] One more that I didn't think about earlier: A while back, we spent quite some time defining the semantics of the various commands in the face of the NBD_CMD_F

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

2016-09-09 Thread Wouter Verhelst
On Fri, Sep 09, 2016 at 04:36:07PM -0400, Josef Bacik wrote: > On 09/09/2016 04:02 PM, Wouter Verhelst wrote: [...] > > I see some practical problems with this: > > - You removed the pid attribute from sysfs (unless you added it back and > > I didn't notice, in which

Re: [Nbd] [RESEND][PATCH 0/5] nbd improvements

2016-09-09 Thread Wouter Verhelst
Hi Josef, On Thu, Sep 08, 2016 at 05:12:05PM -0400, Josef Bacik wrote: > Apologies if you are getting this a second time, it appears vger ate my last > submission. > > -- > > This is a patch series aimed at bringing NBD into 201

Re: [Nbd] [PATCH 0/5] nbd improvements

2016-09-09 Thread Wouter Verhelst
Hi Jens, On Thu, Sep 08, 2016 at 02:13:46PM -0600, Jens Axboe wrote: > Not only are these nice performance improvements, it also cleans up the > code a lot and kills crufty old driver code in favor of using the proper > APIs instead. I have applied the series for 4.9. Please ensure that the comme

Re: [Nbd] [PATCH 2/2] nbd: Disallow ioctls on disconnected block device

2016-07-16 Thread Wouter Verhelst
On Sat, Jul 16, 2016 at 03:38:40PM +0530, Pranay Srivastava wrote: > Okay. So how about we include some negotiated key which goes in with every > request which the server could maintain for clients that can be checked while > resetting the connection with the same server? Wut? > So am I correct t

Re: [Nbd] [PATCH v2 4/5]nbd: make nbd device wait for its users.

2016-06-15 Thread Wouter Verhelst
On Wed, Jun 15, 2016 at 08:30:45AM +0200, Markus Pargmann wrote: > Thanks for the explanations. I think my understanding was off by one ;).. > I didn't realize that the DO_IT thread from the userspace has the block > device open as well. Obviously, otherwise it couldn't do an ioctl() to it :-) >

Re: [Nbd] [PATCH] NBD: replace kill_bdev() with __invalidate_device()

2016-05-24 Thread Wouter Verhelst
On Thu, May 19, 2016 at 08:35:03AM +0200, Markus Pargmann wrote: > Hi Wouter, > > On Sun, May 15, 2016 at 02:55:39PM +0200, Wouter Verhelst wrote: > > Hi Markus, > > > > On Thu, May 12, 2016 at 11:53:01AM +0200, Markus Pargmann wrote: > > > On Thursday 28

Re: [Nbd] [PATCH] NBD: replace kill_bdev() with __invalidate_device()

2016-05-15 Thread Wouter Verhelst
Hi Markus, On Thu, May 12, 2016 at 11:53:01AM +0200, Markus Pargmann wrote: > On Thursday 28 April 2016 18:27:34 Wouter Verhelst wrote: > > However, at some point I agreed with Paul (your predecessor) that when > > this happens due to an error condition (as opposed to it

Re: [Nbd] [PATCH] NBD: replace kill_bdev() with __invalidate_device()

2016-04-28 Thread Wouter Verhelst
On Thu, Apr 28, 2016 at 11:00:20AM +0200, Markus Pargmann wrote: > Hi, > > On Saturday 23 April 2016 07:47:21 Ratna Manoj wrote: > > Thanks for the review. > > > > Atleast for ext4 this crash happens on a sys_umount() call, timing of > > which is not in control of block driver. Block driver cann

Re: [PATCH v2] nbd: Issue a uevent when disconnecting

2016-02-05 Thread Wouter Verhelst
Hi Markus, On Fri, Feb 05, 2016 at 10:03:41AM +0100, Markus Pargmann wrote: > Hi Wouter, > > On Tuesday, February 02, 2016 04:27:00 PM Wouter Verhelst wrote: > > There already is a uevent by default when closing a device upon connect > > of the device. However, the same isn

[PATCH v2] nbd: Issue a uevent when disconnecting

2016-02-02 Thread Wouter Verhelst
moved again. Explicitly issue a "change" uevent to remedy. [v2: incorporate feedback from udev maintainer and make sure that we issue a uevent upon connect as well as disconnect, rather than connect only] Signed-off-by: Wouter Verhelst --- drivers/block/nbd.c | 3 +++ 1 file change

[PATCH] Issue a uevent when disconnecting

2016-01-23 Thread Wouter Verhelst
moved again. Explicitly issue a "change" uevent to remedy. Signed-off-by: Wouter Verhelst --- drivers/block/nbd.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c index e4c5cc1..43dcc12 100644 --- a/drivers/block/nbd.c +++ b/drivers/block/nbd.c @

Re: [Nbd] [PATCH 3/9] nbd: Remove kernel internal header

2015-02-14 Thread Wouter Verhelst
On Thu, Feb 12, 2015 at 09:57:31PM +0100, Markus Pargmann wrote: > The header is not included anywhere. Remove it and include the private > nbd_device struct in nbd.c. It exists mostly for the benefit of userspace trying to speak the NBD protocol. I've stopped trying to depend on it (since nbd-ser

Re: [Nbd] [PATCH 2/9] Documentation: nbd: Add list of module parameters

2015-02-14 Thread Wouter Verhelst
On Thu, Feb 12, 2015 at 09:57:30PM +0100, Markus Pargmann wrote: > +max_part > + Number of partitions per device (default: 0). About that. Wouldn't it be better to change that default? Something like 16 makes more sense to me. -- It is easy to love a country that is famous for chocolate and

Re: [Nbd] [RFC 4/4] nbd: Add support for nbd as root device

2015-01-30 Thread Wouter Verhelst
On Fri, Jan 30, 2015 at 09:04:00AM +0100, Markus Pargmann wrote: > Hi, > > On Fri, Jan 30, 2015 at 12:42:54AM +0100, Wouter Verhelst wrote: > > Not that I'm opposed to this, but you do realize that doing nbd-client > > from initramfs or similar is possi

Re: [Nbd] [RFC 4/4] nbd: Add support for nbd as root device

2015-01-29 Thread Wouter Verhelst
On Tue, Jan 13, 2015 at 02:44:25PM +0100, Markus Pargmann wrote: > Adding support to nbd to use it as a root device. This code essentially > provides a minimal nbd-client implementation within the kernel. It opens > a socket and makes the negotiation with the server. Afterwards it passes > the sock

Re: [Nbd] NBD Maintainer

2015-01-29 Thread Wouter Verhelst
On Wed, Jan 14, 2015 at 07:33:43AM +, Tuomas Räsänen wrote: > Hi > > On Tue Jan 13 21:26:01 2015 GMT+0200, Paul Clements wrote: > > On Tue, Jan 13, 2015 at 11:14 AM, Andrey Utkin > > wrote: > > > Hi Paul, > > > could you please describe > > > > > - how wide is NBD usage today (any estimation

Re: [Nbd] [PATCH] MAINTAINERS: Update NBD maintainer

2015-01-29 Thread Wouter Verhelst
Hi Markus, On Wed, Jan 28, 2015 at 07:35:38PM +0100, Markus Pargmann wrote: > Paul stops maintining NBD and I will take his place from now on. Thanks for stepping in! For clarity: I've been maintaining the userspace component (nbd-client/nbd-server) since about the same time, but am not looking

Re: [Nbd] [PATCH 0/3] NBD fixes for caching and block device flags

2013-02-13 Thread Wouter Verhelst
On Tue, Feb 12, 2013 at 01:43:49PM -0800, Andrew Morton wrote: > On Tue, 12 Feb 2013 17:06:08 +0100 > Paolo Bonzini wrote: > > > The first two patches improve the behavior of NBD with respect to the host > > cache---respectively the server's cache and the client's. The first adds > > support for

[PATCH] Update documentation for nbd, and link to mailinglist

2013-02-13 Thread Wouter Verhelst
documentation for the non-kernel bits should live. Additionally, add a reference to the MAINTAINERS file for the nbd-general mailinglist that is used for discussion of the userland tools and the kernel module already. Cc: Cc: Paul Clements Cc: Andrew Morton Signed-off-by: Wouter Verhelst

Re: [RFC 0/3] Recursive reclaim (on __PF_MEMALLOC)

2007-09-18 Thread Wouter Verhelst
On Mon, Sep 17, 2007 at 10:11:25PM -0700, Daniel Phillips wrote: > On Monday 17 September 2007 20:27, Mike Snitzer wrote: > > > - Statically prove bounded memory use of all code in the writeout > > > path. > > > > > > - Implement any special measures required to be able to make such > > > a

Re: [patch 1/2] m68k: runtime patching infrastructure

2007-06-03 Thread Wouter Verhelst
On Tue, May 29, 2007 at 05:38:18PM -0700, Andrew Morton wrote: [...] > > +struct mod_arch_specific { > > + struct m68k_fixup_info *fixup_start, *fixup_end; > > +}; > > Here we use struct m68k_fixup_info. [...] > > +struct m68k_fixup_info { > > + enum m68k_fixup_type type; > > + void *addr; >

Re: retry [PATCH] partition : add support for sysv68 partitions

2007-05-09 Thread Wouter Verhelst
On Tue, May 08, 2007 at 10:21:12PM +0200, Geert Uytterhoeven wrote: > Hi Philippe, > On Fri, 13 Apr 2007, Philippe De Muyter wrote: > > Add support for the Motorola sysv68 disk partition table (slices in motorola > > doc). > > > > Signed-off-by: Philippe De Muyter <[EMAIL PROTECTED]> > > >

CDROM and harddisk fighting over DMA

2001-03-20 Thread Wouter Verhelst
(I'm not subscribed to linux-kernel, so please CC any answers. TIA) Hello Since I bought my new harddisk (a Maxtor 40GB of about a half year old now), I've had errors over my console like this: hda: timeout waiting for DMA ide_dmaproc: chipset supported ide_dma_timeout func only: 14 hda: irq ti