Re: [PATCH] docs: Tweak location of qemu nbd extensions

2024-08-03 Thread Wouter Verhelst
On Fri, Aug 02, 2024 at 08:36:43AM -0500, Eric Blake wrote: > Upstream QEMU is moving the location of its NBD docs, as of its commit > [1]. Instead of pointing to the raw git source file, point to the > rendered html version built from rST. ACK. Please push as soon as that lands. -- w@

Re: [PATCH v2 5/6] spec: Introduce NBD_FLAG_BLOCK_STATUS_PAYLOAD

2023-03-05 Thread Wouter Verhelst
On Fri, Mar 03, 2023 at 04:40:38PM -0600, Eric Blake wrote: > On Wed, Feb 22, 2023 at 12:05:44PM +0200, Wouter Verhelst wrote: > > On Mon, Nov 14, 2022 at 04:46:54PM -0600, Eric Blake wrote: > > > Simple reply message > > > > > > @@ -1232,6 +1235,19 @

Re: [PATCH v2 3/6] spec: Add NBD_OPT_EXTENDED_HEADERS

2023-03-05 Thread Wouter Verhelst
On Fri, Mar 03, 2023 at 04:36:41PM -0600, Eric Blake wrote: > On Wed, Feb 22, 2023 at 11:49:18AM +0200, Wouter Verhelst wrote: > > On Mon, Nov 14, 2022 at 04:46:52PM -0600, Eric Blake wrote: [...] > > > + Note that even when extended headers are in use, the client MUST be >

Re: [PATCH v2 2/6] spec: Tweak description of maximum block size

2023-03-05 Thread Wouter Verhelst
On Fri, Mar 03, 2023 at 04:26:53PM -0600, Eric Blake wrote: > On Tue, Feb 21, 2023 at 05:21:37PM +0200, Wouter Verhelst wrote: > > Hi Eric, > > > > Busy days, busy times. Sorry about the insane delays here. > > No problem; I've been tackling other things in the m

Re: [PATCH v2 1/6] spec: Recommend cap on NBD_REPLY_TYPE_BLOCK_STATUS length

2023-03-05 Thread Wouter Verhelst
On Fri, Mar 03, 2023 at 04:17:40PM -0600, Eric Blake wrote: > On Fri, Dec 16, 2022 at 10:32:01PM +0300, Vladimir Sementsov-Ogievskiy wrote: > > s-o-b line missed. > > I'm not sure if the NBD project has a strict policy on including one, > but I don't mind adding it. I've never required it, mostly

Re: [PATCH v2 5/6] spec: Introduce NBD_FLAG_BLOCK_STATUS_PAYLOAD

2023-02-22 Thread Wouter Verhelst
On Mon, Nov 14, 2022 at 04:46:54PM -0600, Eric Blake wrote: > Simple reply message > > @@ -1232,6 +1235,19 @@ The field has the following format: >will be faster than a regular write). Clients MUST NOT set the >`NBD_CMD_FLAG_FAST_ZERO` request flag unless this transmission flag >

Re: [PATCH v2 3/6] spec: Add NBD_OPT_EXTENDED_HEADERS

2023-02-22 Thread Wouter Verhelst
On Mon, Nov 14, 2022 at 04:46:52PM -0600, Eric Blake wrote: [...] > @@ -1370,9 +1475,10 @@ of the newstyle negotiation. > Return a list of `NBD_REP_META_CONTEXT` replies, one per context, > followed by an `NBD_REP_ACK` or an error. > > -This option SHOULD NOT be requested unless stru

Re: [PATCH v2 2/6] spec: Tweak description of maximum block size

2023-02-21 Thread Wouter Verhelst
Hi Eric, Busy days, busy times. Sorry about the insane delays here. On Mon, Nov 14, 2022 at 04:46:51PM -0600, Eric Blake wrote: > Commit 9f30fedb improved the spec to allow non-payload requests that > exceed any advertised maximum block size. Take this one step further > by permitting the server

Re: [PATCH] spec: Add NBD_OPT_EXTENDED_HEADERS

2022-03-24 Thread Wouter Verhelst
Hi Eric, Thanks for the ping; it had slipped my mind. On Fri, Dec 03, 2021 at 05:14:34PM -0600, Eric Blake wrote: > Request message > > -The request message, sent by the client, looks as follows: > +The compact request message, sent by the client when extended > +transactions are not negot

Re: [PATCH] spec: Add NBD_OPT_EXTENDED_HEADERS

2021-12-07 Thread Wouter Verhelst
On Mon, Dec 06, 2021 at 05:00:47PM -0600, Eric Blake wrote: > On Mon, Dec 06, 2021 at 02:40:45PM +0300, Vladimir Sementsov-Ogievskiy wrote: > > > Simple reply message > > > > > > The simple reply message MUST be sent by the server in response to all > > > requests if structured replies

Re: Cross-project NBD extension proposal: NBD_INFO_INIT_STATE

2020-02-11 Thread Wouter Verhelst
Hi, On Mon, Feb 10, 2020 at 10:52:55PM +, Richard W.M. Jones wrote: > But anyway ... could a flag indicating that the whole image is sparse > be useful, either as well as NBD_INIT_SPARSE or instead of it? You > could use it to avoid an initial disk trim, which is something that > mke2fs does:

Re: [Qemu-devel] [PATCH 1/1] protocol: Add NBD_CMD_FLAG_FAST_ZERO

2019-08-23 Thread Wouter Verhelst
On Fri, Aug 23, 2019 at 01:58:44PM -0500, Eric Blake wrote: > On 8/23/19 1:48 PM, Wouter Verhelst wrote: > > On Fri, Aug 23, 2019 at 09:34:26AM -0500, Eric Blake wrote: > >> +- bit 4, `NBD_CMD_FLAG_FAST_ZERO`; valid during > >> + `NBD_CMD_WRITE_ZEROES`. If set, but th

Re: [Qemu-devel] [PATCH 1/1] protocol: Add NBD_CMD_FLAG_FAST_ZERO

2019-08-23 Thread Wouter Verhelst
On Fri, Aug 23, 2019 at 09:34:26AM -0500, Eric Blake wrote: > +- bit 4, `NBD_CMD_FLAG_FAST_ZERO`; valid during > + `NBD_CMD_WRITE_ZEROES`. If set, but the server cannot perform the > + write zeroes any faster than it would for an equivalent > + `NBD_CMD_WRITE`, One way of fulfilling the letter

Re: [Qemu-devel] [Nbd] [PATCH] Further tidy-up on block status

2018-05-03 Thread Wouter Verhelst
thought we'd incorporated it already at the time, but apparently not. I still think this is a good idea, and that we should do it. > On 12/14/2016 11:09 AM, Wouter Verhelst wrote: > > > One thing I've been thinking about that we might want to add: > > > > There may be ca

Re: [Qemu-devel] [PATCH 1/4] nbd/server: refactor nbd_negotiate_meta_query for several namespaces

2018-03-21 Thread Wouter Verhelst
On Wed, Mar 21, 2018 at 09:56:36AM -0500, Eric Blake wrote: > no further qemu patches are submitted. Is it worth me proposing a doc > change to demonstrate what the difference would look like, along with > corresponding qemu changes to match, to decide if including it in 2.12 is > worth it? If yo

Re: [Qemu-devel] [Nbd] [PATCH] Further tidy-up on block status

2018-02-28 Thread Wouter Verhelst
Hi, Sorry, I forgot to reply to this earlier. On Fri, Feb 16, 2018 at 10:10:59AM -0600, Eric Blake wrote: > On 02/16/2018 07:53 AM, Vladimir Sementsov-Ogievskiy wrote: > > Good idea. But it would be tricky thing to maintain backward > > compatibility with published versions of virtuozzo product.

Re: [Qemu-devel] drive-mirroring to nbd is failing with multiple parallel jobs (qemu 2.9 -> 2.11)

2018-02-15 Thread Wouter Verhelst
Hi Eric, On Wed, Feb 14, 2018 at 09:11:02AM -0600, Eric Blake wrote: [NBD and keepalive] > This is more food for thought on whether it even makes sense for NBD to > worry about assisting in keepalive matters, or whether it would just be > bloating the protocol. I'm currently leaning towards the l

Re: [Qemu-devel] RFC: Let NBD client request read-only mode

2017-12-01 Thread Wouter Verhelst
On Thu, Nov 30, 2017 at 10:00:46AM -0600, Eric Blake wrote: > On 11/30/2017 09:32 AM, Wouter Verhelst wrote: > > > A client that wants to be read-only, but which does not see server support > > > (in idea 1, the server did not advertise the bit; in idea 2, the serv

Re: [Qemu-devel] RFC: Let NBD client request read-only mode

2017-11-30 Thread Wouter Verhelst
On Wed, Nov 29, 2017 at 08:57:20AM -0600, Eric Blake wrote: > Right now, only the server can choose whether an export is read-only. A > client can always treat an export as read-only by not sending any writes, > but a server has no guarantee that a client will behave that way, and must > assume th

Re: [Qemu-devel] [Nbd] [Qemu-block] How to online resize qemu disk with nbd protocol?

2017-11-23 Thread Wouter Verhelst
Ping. Should I write this up as a proper proposal? On Thu, Nov 16, 2017 at 05:20:29PM +0100, Wouter Verhelst wrote: > On Thu, Nov 16, 2017 at 09:30:41AM -0600, Eric Blake wrote: > > On 11/16/2017 03:51 AM, Wouter Verhelst wrote: > > > > >> I also remember from talki

Re: [Qemu-devel] [Nbd] [Qemu-block] How to online resize qemu disk with nbd protocol?

2017-11-16 Thread Wouter Verhelst
On Thu, Nov 16, 2017 at 09:30:41AM -0600, Eric Blake wrote: > On 11/16/2017 03:51 AM, Wouter Verhelst wrote: > > >> I also remember from talking with Vladimir during KVM Forum last month > >> that one of the shortfalls of the NBD protocol is that you can only ever > &

Re: [Qemu-devel] [Nbd] [Qemu-block] How to online resize qemu disk with nbd protocol?

2017-11-16 Thread Wouter Verhelst
On Tue, Nov 14, 2017 at 01:06:17PM -0600, Eric Blake wrote: > On 11/14/2017 11:37 AM, Wouter Verhelst wrote: > > On Tue, Nov 14, 2017 at 10:41:39AM -0600, Eric Blake wrote: > >> Another thought - with structured replies, we finally have a way to let > >> the client

Re: [Qemu-devel] [Nbd] [Qemu-block] How to online resize qemu disk with nbd protocol?

2017-11-14 Thread Wouter Verhelst
On Tue, Nov 14, 2017 at 10:41:39AM -0600, Eric Blake wrote: > Another thought - with structured replies, we finally have a way to let > the client ask for the server to send resize information whenever the > server wants, rather than having to be polled by a new client request > all the time. This

Re: [Qemu-devel] structured reply behavior for read of 0 bytes

2017-11-05 Thread Wouter Verhelst
Hi Eric, On Fri, Nov 03, 2017 at 05:45:07PM -0500, Eric Blake wrote: > As currently written, structured reply is documented as: > > > NBD_REPLY_TYPE_OFFSET_DATA (1) > > > > This chunk type is in the content chunk category. length MUST be at least > > 9. It represents the contents of length - 8 b

Re: [Qemu-devel] nbd structured reply

2017-09-23 Thread Wouter Verhelst
On Fri, Sep 22, 2017 at 05:57:07PM +0300, Vladimir Sementsov-Ogievskiy wrote: > The obvious behavior of client is to fail the whole read if it received one > error chunk. Not necessarily. If a user-space program requests to read X bytes of data, but there is an error at X-N, then the obvious way

Re: [Qemu-devel] [Nbd] [Qemu-block] How to online resize qemu disk with nbd protocol?

2017-01-25 Thread Wouter Verhelst
Hi Eric, On Mon, Jan 23, 2017 at 08:54:48AM -0600, Eric Blake wrote: > On 01/22/2017 05:43 AM, Wouter Verhelst wrote: > > > > Having given this some more thought, I'm not entirely sure anymore that > > an active resize from the NBD layer is necessarily a layering vio

Re: [Qemu-devel] [Nbd] [Qemu-block] How to online resize qemu disk with nbd protocol?

2017-01-22 Thread Wouter Verhelst
#x27; I read is not always correct. Sometimes the > bytes are repeating, for instance 1073741824 (1GB) became 1073741824073741824 > ... > > Could you help me figure out what went wrong? > > > Regards,  > Bob > > 2017-01-18 16:01 GMT+08:00 Wouter Verhelst : > &

Re: [Qemu-devel] [Nbd] [PATCH] Further tidy-up on block status

2017-01-21 Thread Wouter Verhelst
On Fri, Jan 20, 2017 at 01:35:10PM -0600, Eric Blake wrote: > On 01/20/2017 12:00 PM, Alex Bligh wrote: > > > >> On 20 Jan 2017, at 17:04, Vladimir Sementsov-Ogievskiy > >> wrote: > >> > >> About extents: is 32bit length enough? We will have to send 4096 for empty > >> 16tb disk.. > > > > The

Re: [Qemu-devel] [Nbd] [Qemu-block] How to online resize qemu disk with nbd protocol?

2017-01-18 Thread Wouter Verhelst
On Mon, Jan 16, 2017 at 01:36:21PM -0600, Eric Blake wrote: > Maybe the structured reply proposal can be extended into this (reserve a > "reply" header that can be issued as many times as desired by the server > without the client ever having issued the request first, and where the > reply never us

Re: [Qemu-devel] [Nbd] [Qemu-block] How to online resize qemu disk with nbd protocol?

2017-01-14 Thread Wouter Verhelst
On Thu, Jan 12, 2017 at 06:56:42PM +, Alex Bligh wrote: > My preferred way to do this would be essentially to allow NBD_OPT_INFO > to be sent (wrapped appropriately) during the main transmission phase. > That would allow *any* INFO stuff to be reread. Can you go into a bit more detail how you'

Re: [Qemu-devel] [Nbd] [Qemu-block] How to online resize qemu disk with nbd protocol?

2017-01-14 Thread Wouter Verhelst
Hi Eric, (side note: my mutt tells me that the signature on your message does not validate. Not sure what's going on, but something you might want to look into...) On Thu, Jan 12, 2017 at 12:45:56PM -0600, Eric Blake wrote: > For resize smaller, things are a lot trickier - how do you block access

Re: [Qemu-devel] [Nbd] [PATCH] Further tidy-up on block status

2016-12-27 Thread Wouter Verhelst
Hi Vladimir, On Mon, Dec 26, 2016 at 05:52:54PM +0300, Vladimir Sementsov-Ogievskiy wrote: > Shouldn't we add some flags to REP_META_CONTEXT, for client to be insure, is > returned string a direct context name or some kind of wildcard? Just a flags > field, with one flag defined for now: NBD_REP_M

Re: [Qemu-devel] [Nbd] [PATCH] Further tidy-up on block status

2016-12-17 Thread Wouter Verhelst
On Fri, Dec 16, 2016 at 04:25:27PM +, Alex Bligh wrote: > > > On 16 Dec 2016, at 15:52, Wouter Verhelst wrote: > > > > On Thu, Dec 15, 2016 at 05:34:48PM +, Alex Bligh wrote: > >> > >>> On 15 Dec 2016, at 16:49, Wouter Verhelst wrote: >

Re: [Qemu-devel] [Nbd] [PATCH] Further tidy-up on block status

2016-12-16 Thread Wouter Verhelst
On Thu, Dec 15, 2016 at 05:34:48PM +, Alex Bligh wrote: > > > On 15 Dec 2016, at 16:49, Wouter Verhelst wrote: > > > >> Because the namespaces and leaf-names are already restricted to > >> non-whitespace characters. I thought having tabs, line feeds, >

Re: [Qemu-devel] [Nbd] [PATCH] Further tidy-up on block status

2016-12-15 Thread Wouter Verhelst
On Thu, Dec 15, 2016 at 04:32:23PM +, Alex Bligh wrote: > Wouter, > > > This reads a bit awkward. I would do: > > > > s/save that:/except as explained below/ > > Possibly a British English thing. Will fix. It's mostly that the "save that:" suggests to me that the exception follows immediate

Re: [Qemu-devel] [Nbd] [PATCH] Further tidy-up on block status

2016-12-15 Thread Wouter Verhelst
Hi Alex, On Thu, Dec 15, 2016 at 10:04:05AM +, Alex Bligh wrote: > > > On 14 Dec 2016, at 20:18, Wouter Verhelst wrote: > > > >> +* the server MAY return a context consisting of a namespace and > >> + a colon only (i.e. omitting the leaf-name)

Re: [Qemu-devel] [Nbd] [PATCH] Further tidy-up on block status

2016-12-14 Thread Wouter Verhelst
On Wed, Dec 14, 2016 at 06:51:48PM +, Alex Bligh wrote: > > > On 14 Dec 2016, at 18:18, Alex Bligh wrote: > > > > Let me have a go at a change. > > OK I've pushed a change. I reordered a few bits (to put the > base:allocation next to the stuff that talks about metadata > queries at the star

Re: [Qemu-devel] [Nbd] [PATCH] Further tidy-up on block status

2016-12-14 Thread Wouter Verhelst
On Wed, Dec 14, 2016 at 07:01:15PM +, Alex Bligh wrote: > Wouter, > > (Our mails crossed and I've actually pushed something, but no matter) > > > On 14 Dec 2016, at 18:49, Wouter Verhelst wrote: > > > > What I was trying to say is that I think the r

Re: [Qemu-devel] [Nbd] [Qemu-block] [PATCH] doc: Propose NBD_FLAG_INIT_ZEROES extension

2016-12-14 Thread Wouter Verhelst
Hi Eric, On Wed, Dec 14, 2016 at 10:37:43AM -0600, Eric Blake wrote: > On 12/14/2016 02:22 AM, Wouter Verhelst wrote: > > Hi Eric, > > > > On Tue, Dec 13, 2016 at 04:36:08PM -0600, Eric Blake wrote: > >> On 12/13/2016 06:18 AM, Wouter Verhelst wrote: > >&

Re: [Qemu-devel] [Nbd] [PATCH] Further tidy-up on block status

2016-12-14 Thread Wouter Verhelst
On Wed, Dec 14, 2016 at 06:18:58PM +, Alex Bligh wrote: > > > On 14 Dec 2016, at 18:13, Wouter Verhelst wrote: > > > > Instead, I would suggest that the spec leave it up to the namespace spec > > to define what gets listed when. The namespace should still give

Re: [Qemu-devel] [Nbd] [PATCH] Further tidy-up on block status

2016-12-14 Thread Wouter Verhelst
On Wed, Dec 14, 2016 at 08:55:56PM +0300, Vladimir Sementsov-Ogievskiy wrote: > 14.12.2016 20:36, Alex Bligh wrote: > >> On 14 Dec 2016, at 17:05, Vladimir Sementsov-Ogievskiy > >> wrote: > >>> However, this raises another question. Wouter deliberately made the > >>> query format freeform so that

Re: [Qemu-devel] [Nbd] [PATCH] Further tidy-up on block status

2016-12-14 Thread Wouter Verhelst
On Wed, Dec 14, 2016 at 03:08:40PM +, Alex Bligh wrote: > (NB: I've already applied this and pushed it) Thanks. > * Change NBD_OPT_LIST_METADATA etc. to explicitly send a list of queries > and add a count of queries so we can extend the command later (rather than > rely on the length of o

Re: [Qemu-devel] [Nbd] [Qemu-block] [PATCH] doc: Propose NBD_FLAG_INIT_ZEROES extension

2016-12-14 Thread Wouter Verhelst
Hi Eric, On Tue, Dec 13, 2016 at 04:36:08PM -0600, Eric Blake wrote: > On 12/13/2016 06:18 AM, Wouter Verhelst wrote: > > On Tue, Dec 13, 2016 at 08:38:12AM +0100, Kevin Wolf wrote: > >> Am 12.12.2016 um 19:12 hat Wouter Verhelst geschrieben: > >>> I'm not op

Re: [Qemu-devel] [Nbd] [PATCH v5] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-12-13 Thread Wouter Verhelst
On Tue, Dec 13, 2016 at 03:24:25PM +, Alex Bligh wrote: > Wouter, > > The spec should only specify how to select a particular namespace, and > > then leave all parsing rules of query strings up to the namespace > > specification. > > I'm not quite sure why you want that, Let's say you want to

Re: [Qemu-devel] [Nbd] [PATCH v5] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-12-13 Thread Wouter Verhelst
Alex, (leaving out obvious grammar/language fixes that I have no problem with) On Tue, Dec 13, 2016 at 02:09:19PM +, Alex Bligh wrote: > Wouter, > > Some comments below: > > > On 12 Dec 2016, at 20:43, Wouter Verhelst wrote: > > > +## Metadata q

Re: [Qemu-devel] [Nbd] [PATCH v5] doc: Add NBD_CMD_BLOCK_STATUS extension (part 2)

2016-12-13 Thread Wouter Verhelst
On Tue, Dec 13, 2016 at 02:12:48PM +, Alex Bligh wrote: > > > On 12 Dec 2016, at 20:43, Wouter Verhelst wrote: > > > > +- `NBD_OPT_SET_META_CONTEXT` (11) > > + > > +Change the set of active metadata contexts. Issuing this command > > +replac

Re: [Qemu-devel] [Nbd] [Qemu-block] [PATCH] doc: Propose NBD_FLAG_INIT_ZEROES extension

2016-12-13 Thread Wouter Verhelst
On Tue, Dec 13, 2016 at 08:38:12AM +0100, Kevin Wolf wrote: > Am 12.12.2016 um 19:12 hat Wouter Verhelst geschrieben: > > I'm not opposed to this proposal, per se, but there seems to be some > > disagreement (by Kevin, for instance) on whether this extension is at > >

Re: [Qemu-devel] [Nbd] [PATCH v5] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-12-13 Thread Wouter Verhelst
On Tue, Dec 13, 2016 at 10:37:00AM +, Stefan Hajnoczi wrote: > On Mon, Dec 12, 2016 at 09:43:13PM +0100, Wouter Verhelst wrote: > > +- `NBD_OPT_LIST_META_CONTEXT` (10) > > + > > +Return a list of `NBD_REP_META_CONTEXT` replies, one per context, > > +followe

Re: [Qemu-devel] [Nbd] [PATCH v4] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-12-12 Thread Wouter Verhelst
On Mon, Dec 12, 2016 at 09:26:15PM +0100, Wouter Verhelst wrote: > > Do we still want to require servers to always send 16 extents (when not > > limited to exactly 1), or is it better to just state that as long as the > > server sends at least one extent (so that the client

[Qemu-devel] [Nbd] [PATCH v5] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-12-12 Thread Wouter Verhelst
v4 -> v5: - Editorial improvements as suggested by Eric (with the exception of absence/absense, which would really need to be done on master if valid) - Rework so that the NBD_STATE_* constants are defined in the "Metadata contexts" section *only*, where the `base:allocation` context is defi

Re: [Qemu-devel] [Nbd] [PATCH v4] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-12-12 Thread Wouter Verhelst
On Mon, Dec 12, 2016 at 01:58:26PM -0600, Eric Blake wrote: > On 12/12/2016 12:21 PM, Wouter Verhelst wrote: > > diff from v3 to v4: > > - Remove some repetitive wording (some sections were written more than > > once); > > - Rework the text to remove all lingerin

[Qemu-devel] [PATCH v4] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-12-12 Thread Wouter Verhelst
diff from v3 to v4: - Remove some repetitive wording (some sections were written more than once); - Rework the text to remove all lingering remains of the "extension" section that isn't being used anymore (the current version should therefore read much easier) - Rename "BASE:allocation" to `b

Re: [Qemu-devel] [Nbd] [PATCH] doc: Propose NBD_FLAG_INIT_ZEROES extension

2016-12-12 Thread Wouter Verhelst
Hi Eric, I'm not opposed to this proposal, per se, but there seems to be some disagreement (by Kevin, for instance) on whether this extension is at all useful. If you can clarify that, I'll be happy to merge it. On Mon, Dec 05, 2016 at 05:42:35PM -0600, Eric Blake wrote: > While not directly rel

Re: [Qemu-devel] [Nbd] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-12-08 Thread Wouter Verhelst
On Thu, Dec 08, 2016 at 03:39:19AM +, Alex Bligh wrote: > > > On 2 Dec 2016, at 18:45, Alex Bligh wrote: > > > > Thanks. That makes sense - or enough sense for me to carry on commenting! > > > I finally had some time to go through this extension in detail. Rather > than comment on all the

Re: [Qemu-devel] [Nbd] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-12-06 Thread Wouter Verhelst
Hi John Sorry for the late reply; weekend was busy, and so was monday. On Fri, Dec 02, 2016 at 03:39:08PM -0500, John Snow wrote: > On 12/02/2016 01:45 PM, Alex Bligh wrote: > > John, > > > >>> +Some storage formats and operations over such formats express a > >>> +concept of data dirtiness. Whe

Re: [Qemu-devel] [Nbd] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-12-02 Thread Wouter Verhelst
Hi Vladimir, On Thu, Dec 01, 2016 at 02:26:28PM +0300, Vladimir Sementsov-Ogievskiy wrote: > 01.12.2016 13:14, Wouter Verhelst wrote: [...] > > -- `NBD_ALLOC_ADD_CONTEXT` (2): the list of allocation contexts > > +- `NBD_META_ADD_CONTEXT` (2): the list of me

Re: [Qemu-devel] [Nbd] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-12-01 Thread Wouter Verhelst
Here's another update. Changes since previous version: - Rename "allocation context" to "metadata context" - Stop making metadata context 0 be special; instead, name it "BASE:allocation" and allow it to be selected like all other contexts. - Clarify in a bit more detail when a server MAY omit me

Re: [Qemu-devel] [Nbd] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-11-29 Thread Wouter Verhelst
On Tue, Nov 29, 2016 at 06:07:56PM +0300, Vladimir Sementsov-Ogievskiy wrote: > 29.11.2016 17:52, Alex Bligh wrote: > > Vladimir, > >> if the bitmap is 010101010101 we will have too many descriptors. > >> For example, 16tb disk, 64k granularity -> 2G of descriptors > >> payload. > > Yep. And the co

Re: [Qemu-devel] [Nbd] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-11-29 Thread Wouter Verhelst
Hi Vladimir, On Tue, Nov 29, 2016 at 03:41:10PM +0300, Vladimir Sementsov-Ogievskiy wrote: > Hi, > > 29.11.2016 13:50, Wouter Verhelst wrote: > > Hi, > > On Mon, Nov 28, 2016 at 06:33:24PM +0100, Wouter Verhelst wrote: > > However, I'm arguin

Re: [Qemu-devel] [Nbd] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-11-29 Thread Wouter Verhelst
Hi, On Mon, Nov 28, 2016 at 06:33:24PM +0100, Wouter Verhelst wrote: > However, I'm arguing that if we're going to provide information about > snapshots, we should be able to properly refer to these snapshots from > within an NBD context. My previous mail suggested adding a ne

Re: [Qemu-devel] [Nbd] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-11-28 Thread Wouter Verhelst
Hi Stefan, On Mon, Nov 28, 2016 at 11:19:44AM +, Stefan Hajnoczi wrote: > On Sun, Nov 27, 2016 at 08:17:14PM +0100, Wouter Verhelst wrote: > > Quickly: the reason I haven't merged this yes is twofold: > > - I wasn't thrilled with the proposal at the time. It felt

Re: [Qemu-devel] [Nbd] [PATCH v3] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-11-27 Thread Wouter Verhelst
; a list of extents in the response of NBD_CMD_BLOCK_STATUS command, > instead of a bitmap. > > CC: Pavel Borzenkov > CC: Denis V. Lunev > CC: Wouter Verhelst > CC: Paolo Bonzini > CC: Kevin Wolf > CC: Stefan Hajnoczi > Signed-off-by: Eric Blake > Signed-off

Re: [Qemu-devel] [Nbd] [PATCH] proto: add 'shift' extension.

2016-09-26 Thread Wouter Verhelst
On Mon, Sep 26, 2016 at 03:21:46PM -0500, Eric Blake wrote: > I'd much rather support a single flag that says to zero the entire disk > than to introduce stateful variable-amount shifting. That's almost exactly the opposite of what I said :) Now, I don't feel very strong either way, but what matt

Re: [Qemu-devel] write_zeroes/trim on the whole disk

2016-09-24 Thread Wouter Verhelst
On Sat, Sep 24, 2016 at 11:19:53PM +0300, Vladimir Sementsov-Ogievskiy wrote: > On 24.09.2016 21:24, Alex Bligh wrote: > > > On 24 Sep 2016, at 18:47, Vladimir Sementsov-Ogievskiy > > > wrote: > > > > > > I just wanted to say, that if we want a possibility of clearing the whole > > > disk in on

Re: [Qemu-devel] [Nbd] write_zeroes/trim on the whole disk

2016-09-24 Thread Wouter Verhelst
On Sat, Sep 24, 2016 at 11:31:25AM +0100, Alex Bligh wrote: > > > On 23 Sep 2016, at 22:21, Wouter Verhelst wrote: > > > > On Fri, Sep 23, 2016 at 02:00:06PM -0500, Eric Blake wrote: > >> My preference would be a new flag to the existing commands, with > >&g

Re: [Qemu-devel] write_zeroes/trim on the whole disk

2016-09-23 Thread Wouter Verhelst
On Fri, Sep 23, 2016 at 02:00:06PM -0500, Eric Blake wrote: > My preference would be a new flag to the existing commands, with > explicit documentation that 0 offset and 0 length must be used with that > flag, when requesting a full-device wipe. Alternatively, what about a flag that says "if you u

Re: [Qemu-devel] [Nbd] [PATCH v4 04/11] nbd: Improve server handling of bogus commands

2016-06-15 Thread Wouter Verhelst
On Wed, Jun 15, 2016 at 11:27:21AM +0100, Alex Bligh wrote: > Perhaps this should read "If an error occurs, the server MUST either initiate > a hard disconnect before the entire payload has been sent or > set the appropriate code in the error field and send the response header > without any payload

Re: [Qemu-devel] [Nbd] [PATCH v4 04/11] nbd: Improve server handling of bogus commands

2016-06-15 Thread Wouter Verhelst
On Wed, Jun 15, 2016 at 09:05:22AM +0200, Wouter Verhelst wrote: > There are more clients than the Linux and qemu ones, but I think it's > fair to say that those two are the most important ones. If they agree > that a read reply which errors should come without payload, then I thi

Re: [Qemu-devel] [Nbd] [PATCH v4 04/11] nbd: Improve server handling of bogus commands

2016-06-15 Thread Wouter Verhelst
On Tue, Jun 14, 2016 at 04:02:15PM +0100, Alex Bligh wrote: > > On 14 Jun 2016, at 14:32, Paolo Bonzini wrote: > > > > > On 13/06/2016 23:41, Alex Bligh wrote: > >> That's one of the reasons that there is a proposal to add > >> STRUCTURED_READ to the spec (although I still haven't had time to >

Re: [Qemu-devel] [Nbd] [PATCH v4 04/11] nbd: Improve server handling of bogus commands

2016-06-15 Thread Wouter Verhelst
On Mon, Jun 13, 2016 at 10:41:05PM +0100, Alex Bligh wrote: > For amusement value, the non-threaded handler (which is not used > any more) does not send any payload on an error: > https://github.com/yoe/nbd/blob/master/nbd-server.c#L1734 nbd-server used to just drop the connection on read error.

Re: [Qemu-devel] [Nbd] [PULL 23/28] nbd: always query export list in fixed new style protocol

2016-05-21 Thread Wouter Verhelst
On Tue, May 17, 2016 at 10:50:01AM -0600, Eric Blake wrote: > Option 2: An alternative solution would be to allow nbdkit to fail > NBD_OPT_LIST with NBD_REP_ERR_UNSUP, at which point qemu client of 2.6 > should just ignore the failure and proceed on to NBD_OPT_EXPORT_NAME. > It is the fact that it

Re: [Qemu-devel] [Nbd] [PATCH] nbd: fix trim/discard commands with a length bigger than NBD_MAX_BUFFER_SIZE

2016-05-11 Thread Wouter Verhelst
On Tue, May 10, 2016 at 04:38:29PM +0100, Alex Bligh wrote: > On 10 May 2016, at 16:29, Eric Blake wrote: > > So the kernel is currently one of the clients that does NOT honor block > > sizes, and as such, servers should be prepared for ANY size up to > > UINT_MAX (other than DoS handling). > > O

Re: [Qemu-devel] [Nbd] [PATCH] nbd: fix trim/discard commands with a length bigger than NBD_MAX_BUFFER_SIZE

2016-05-11 Thread Wouter Verhelst
On Tue, May 10, 2016 at 09:29:23AM -0600, Eric Blake wrote: > On 05/10/2016 09:08 AM, Alex Bligh wrote: > > Eric, > > > >> Hmm. The current wording of the experimental block size additions does > >> NOT allow the client to send a NBD_CMD_TRIM with a size larger than the > >> maximum NBD_CMD_WRITE:

Re: [Qemu-devel] [Nbd] [PATCH] nbd: fix trim/discard commands with a length bigger than NBD_MAX_BUFFER_SIZE

2016-05-11 Thread Wouter Verhelst
On Tue, May 10, 2016 at 04:08:50PM +0100, Alex Bligh wrote: > What surprises me is that a release kernel is using experimental > NBD extensions; there's no guarantee these won't change. Or does > fstrim work some other way? What makes you say NBD_CMD_TRIM is experimental? It's been implemented for

Re: [Qemu-devel] [Nbd] question on ioctl NBD_SET_FLAGS

2016-04-20 Thread Wouter Verhelst
Hi Eric, On Wed, Apr 20, 2016 at 09:42:02AM -0600, Eric Blake wrote: [...] > But in 3.9, the overlap bug was still present, and the set of global > flags had grown to include NBD_FLAG_NO_ZEROS (commit 038e066), which > overlaps with NBD_FLAG_READ_ONLY. Ouch. This means that a client > talking to

Re: [Qemu-devel] [Nbd] [PATCHv8] Improve documentation for TLS

2016-04-12 Thread Wouter Verhelst
On Tue, Apr 12, 2016 at 01:57:25PM +0100, Alex Bligh wrote: > > On 12 Apr 2016, at 13:40, Wouter Verhelst wrote: > > > Right, that sounds good. > > Great. I may look at that when the other doc patches are applied. > > On which note, back to $subject, how is PATCH

Re: [Qemu-devel] [Nbd] [PATCHv8] Improve documentation for TLS

2016-04-12 Thread Wouter Verhelst
On Tue, Apr 12, 2016 at 10:53:57AM +0100, Alex Bligh wrote: > Wouter, > > On 12 Apr 2016, at 10:20, Wouter Verhelst wrote: > > > To summarize, there are three ways for the connection to end: > > > > - The client wishes to end the session, and sends the approp

Re: [Qemu-devel] [Nbd] [PATCHv8] Improve documentation for TLS

2016-04-12 Thread Wouter Verhelst
On Tue, Apr 12, 2016 at 08:47:49AM +0100, Alex Bligh wrote: > > On 12 Apr 2016, at 07:01, Wouter Verhelst wrote: > > > hat doesn't mean OPT_ABORT not having a reply is necessarily a good > > idea. Since it's only used by reference nbd-client in just one use case &

Re: [Qemu-devel] [Nbd] [PATCHv8] Improve documentation for TLS

2016-04-11 Thread Wouter Verhelst
On Mon, Apr 11, 2016 at 09:34:44PM +0100, Alex Bligh wrote: > Eric, > > On 11 Apr 2016, at 21:14, Eric Blake wrote: > > Current qemu NBD server implementation does NOT send a reply to > > NBD_OPT_ABORT, but immediately closes the connection. I don't know if > > that is a bug in qemu (especially g

Re: [Qemu-devel] [Nbd] [PATCHv8] Improve documentation for TLS

2016-04-10 Thread Wouter Verhelst
Mostly there. Final note: On Sun, Apr 10, 2016 at 01:47:32PM +0100, Alex Bligh wrote: > diff --git a/doc/proto.md b/doc/proto.md > index f117394..5005552 100644 > --- a/doc/proto.md > +++ b/doc/proto.md > @@ -195,6 +195,13 @@ request before sending the next one of the same type. > The server MAY

Re: [Qemu-devel] [Nbd] [PATCHv3] Improve documentation for TLS

2016-04-09 Thread Wouter Verhelst
On Sat, Apr 09, 2016 at 12:21:03PM +0100, Alex Bligh wrote: > An alternative route would be to delete OPTIONALTLS, and make some of > the MUST requirements in SELECTIVETLS say "MUST xyz unless there are > no TLS-only exports". However, this makes it rather harder to read, > so I described that case

Re: [Qemu-devel] [Nbd] [PATCHv3] Improve documentation for TLS

2016-04-09 Thread Wouter Verhelst
On Sat, Apr 09, 2016 at 11:26:23AM +0100, Alex Bligh wrote: > > On 9 Apr 2016, at 11:11, Wouter Verhelst wrote: > > Since you say zero here, how is it different from OPTIONALTLS? > > > > If "not at all", just drop optional. > > As per previous message, b

Re: [Qemu-devel] [Nbd] [PATCH] Improve documentation for TLS

2016-04-09 Thread Wouter Verhelst
On Sat, Apr 09, 2016 at 11:04:09AM +0100, Alex Bligh wrote: [...] > > [...] > >> +The server MUST NOT send `NBD_REP_ERR_TLS_REQD` in reply to > >> +any command if TLS has already been neogitated. The server > > > > negotiated > > I'd make sure you're looking at the latest version as Eagle Eyed Er

Re: [Qemu-devel] [Nbd] [PATCH] Improve documentation for TLS

2016-04-09 Thread Wouter Verhelst
On Sat, Apr 09, 2016 at 11:05:16AM +0100, Alex Bligh wrote: > > On 9 Apr 2016, at 10:50, Wouter Verhelst wrote: > > > So if I want to swap to qemu-nbd, I cannot also have encrypted > > connections to the same server. Got it. > > AFAIK qemu-nbd only supports a

Re: [Qemu-devel] [Nbd] [RFC PATCH 00/18] NBD protocol additions

2016-04-09 Thread Wouter Verhelst
On Fri, Apr 08, 2016 at 04:05:40PM -0600, Eric Blake wrote: > This series is for qemu 2.7, and will probably need some rework > especially since some of it is trying to implement features > that are still marked experimental in upstream NBD. > > Included are some interoperability bug fixes, code c

Re: [Qemu-devel] [Nbd] [PATCHv3] Improve documentation for TLS

2016-04-09 Thread Wouter Verhelst
On Thu, Apr 07, 2016 at 07:32:47PM +0100, Alex Bligh wrote: [...] > +### Server-side requirements > + > +There are four modes of operation for a server. The > +server MUST support one of these modes. > + > +* The server operates entirely without TLS ('NOTLS'); OR > + > +* The server makes TLS avail

Re: [Qemu-devel] [Nbd] [PATCH] Improve documentation for TLS

2016-04-09 Thread Wouter Verhelst
On Thu, Apr 07, 2016 at 08:31:23AM -0600, Eric Blake wrote: > > +The FORCEDTLS mode of operation has an implementation problem in > > +that the client MAY legally simply send a `NBD_OPT_EXPORT_NAME` > > +to enter transmission mode without previously sending any options. > > +Therefore, if a server

Re: [Qemu-devel] [Nbd] [PATCH] Improve documentation for TLS

2016-04-09 Thread Wouter Verhelst
On Thu, Apr 07, 2016 at 02:56:48PM +0100, Daniel P. Berrange wrote: > I don't really agree that there's a use case of mixing > tls & non-tls exports in the same server. There is: swap-on-NBD and TLS do not mix. Without special handling, swapping to the network is prone to deadlocks, because the m

Re: [Qemu-devel] [Nbd] [PATCH] Improve documentation for TLS

2016-04-09 Thread Wouter Verhelst
On Thu, Apr 07, 2016 at 12:35:59PM +0100, Alex Bligh wrote: > * Call out TLS into a separate section > > * Add details of the TLS protocol itself > > * Emphasise that actual TLS session initiation (i.e. the TLS handshake) can > be initiated from either side (as required by the TLS standard I be

Re: [Qemu-devel] [Nbd] [PATCH v2] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-04-09 Thread Wouter Verhelst
On Thu, Apr 07, 2016 at 10:10:58AM -0600, Eric Blake wrote: > On 04/07/2016 04:38 AM, Vladimir Sementsov-Ogievskiy wrote: > > On 05.04.2016 16:43, Paolo Bonzini wrote: > >> > >> On 05/04/2016 06:05, Kevin Wolf wrote: > >>> The options I can think of is adding a request field "max number of > >>> de

Re: [Qemu-devel] [Nbd] [PATCH v2] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-04-05 Thread Wouter Verhelst
On Mon, Apr 04, 2016 at 05:32:34PM -0600, Eric Blake wrote: > On 04/04/2016 05:08 PM, Wouter Verhelst wrote: > > On Mon, Apr 04, 2016 at 10:54:02PM +0300, Denis V. Lunev wrote: > >> saying about dirtiness, we would soon come to the fact, that > >> we can have several

Re: [Qemu-devel] [Nbd] [PATCHv2] Amend NBD_OPT_SELECT (now NBD_OPT_INFO) and NBD_OPT_GO documentation

2016-04-05 Thread Wouter Verhelst
On Tue, Apr 05, 2016 at 09:42:26PM +0100, Alex Bligh wrote: > Amend the NBD_OPT_SELECT and NBD_OPT_GO documentation as > follows: > > * Change NBD_OPT_SELECT to be called NBD_OPT_INFO > > * Remove the 'selection' aspect of that command, so that > it now merely returns information. This is to av

Re: [Qemu-devel] [Nbd] [PATCHv5] Improve the documentation of NBD_CMD_FLUSH and NBD_CMD_FLAG_FUA.

2016-04-05 Thread Wouter Verhelst
On Tue, Apr 05, 2016 at 05:43:14PM +0100, Alex Bligh wrote: > Improve the documentation of NBD_CMD_FLUSH and NBD_CMD_FLAG_FUA. Specifically > the latter may be set on any command, and its semantics on commands other > than NBD_CMD_WRITE need explaining. Further, explain how these relate to > reorde

Re: [Qemu-devel] [Nbd] [PATCH v2] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-04-05 Thread Wouter Verhelst
On Tue, Apr 05, 2016 at 08:14:01AM -0600, Eric Blake wrote: > On 04/05/2016 03:24 AM, Markus Pargmann wrote: > > >> +requested. > >> + > >> +The client SHOULD NOT read from an area that has both > >> +`NBD_STATE_HOLE` set and `NBD_STATE_ZERO` clear. > > > > Why not? If we don't ex

Re: [Qemu-devel] [Nbd] [PATCH v2] doc: Allow NBD_CMD_FLAG_NO_HOLE during NBD_CMD_WRITE

2016-04-05 Thread Wouter Verhelst
On Tue, Apr 05, 2016 at 10:43:14AM -0600, Eric Blake wrote: > On 04/05/2016 03:38 AM, Markus Pargmann wrote: > > Hi, > > > > On Monday 04 April 2016 16:15:43 Eric Blake wrote: > >> qemu already has an existing server implementation option that will > >> explicitly search the payload of NBD_CMD_WRI

Re: [Qemu-devel] [Nbd] [PATCH v2] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-04-05 Thread Wouter Verhelst
On Mon, Apr 04, 2016 at 05:32:34PM -0600, Eric Blake wrote: > On 04/04/2016 05:08 PM, Wouter Verhelst wrote: > > On Mon, Apr 04, 2016 at 10:54:02PM +0300, Denis V. Lunev wrote: > >> saying about dirtiness, we would soon come to the fact, that > >> we can have several

Re: [Qemu-devel] [Nbd] [PATCH v2] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-04-04 Thread Wouter Verhelst
On Mon, Apr 04, 2016 at 10:54:02PM +0300, Denis V. Lunev wrote: > saying about dirtiness, we would soon come to the fact, that > we can have several dirtiness states regarding different > lines of incremental backups. This complexity is hidden > inside QEMU and it would be very difficult to publish

Re: [Qemu-devel] [Nbd] [PATCH v2] doc: Add NBD_CMD_BLOCK_STATUS extension

2016-04-04 Thread Wouter Verhelst
Hi, Need to look into this in some detail, for which I don't have the time (or the non-tiredness ;-) right now, but these two caught my eye: On Mon, Apr 04, 2016 at 10:39:10AM -0600, Eric Blake wrote: [...] > +* `NBD_REPLY_TYPE_BLOCK_STATUS` > + > +*length* MUST be a positive integer multiple

Re: [Qemu-devel] [Nbd] [PATCH 2/2] NBD proto: add GET_LBA_STATUS extension

2016-04-04 Thread Wouter Verhelst
On Mon, Apr 04, 2016 at 12:18:37PM +0200, Markus Pargmann wrote: > Hi, > > back from my easter vacation. A bit surprised to find 200 mails in the > NBD mailing list ;). I'm sure you were :-) [...] > > Yes. This has been discussed on the nbd-general list in the past. There > > is also the (signif

Re: [Qemu-devel] [Nbd] [PATCHv2] Improve documentation of FUA and FLUSH

2016-04-02 Thread Wouter Verhelst
On Fri, Apr 01, 2016 at 10:54:42AM +0100, Alex Bligh wrote: > Improve the documentation of NBD_CMD_FLUSH and NBD_CMD_FLAG_FUA. Specifically > the latter may be set on any command, and its semantics on commands other > than NBD_CMD_WRITE need explaining. Further, explain how these relate to > reorde

  1   2   >