Re: [Libguestfs] [PATCH libnbd 0/4] Miscellaneous Rust cleanups

2023-10-18 Thread Eric Blake
gressions. For the series: Reviewed-by: Eric Blake There is an XXX about displying links rather than just italicized texts for pod L<> constructs linking to non-libnbd pages, which can be solved later, but your approach for now is definitely better than plain text. -- Eric Blake, Principal S

Re: [Libguestfs] [libnbd PATCH 2/2] info: Show human sizes for block_size values

2023-10-18 Thread Eric Blake
On Sun, Oct 08, 2023 at 11:34:05AM +0200, Laszlo Ersek wrote: > On 10/7/23 11:27, Richard W.M. Jones wrote: > > On Fri, Oct 06, 2023 at 10:18:09AM -0500, Eric Blake wrote: > >> Adding a human-readable size for block constraints is useful. For: > >> > >>

Re: [Libguestfs] [nbdkit] Two POSIX questions for you ...

2023-10-16 Thread Eric Blake
of "tail" and "head" for fetching > the right line. Incredibly slow, of course. > > ... I'm sure stackoverflow has further / better ideas for emulating > arrays in the POSIX shell. > > And then, because keeping the current (fast, but

Re: [Libguestfs] [nbdkit] Two POSIX questions for you ...

2023-10-16 Thread Eric Blake
ot; etc. I'm not sure how that would > > lead to the specific error message, though. It's not just that loop in wrapper.c, but also the loop in server/main.c that is seeing --run as a non-option, at which point the plugin complains that the config callback is being reached too many tim

[Libguestfs] [libnbd PATCH 0/2] Improve nbdinfo display of block constraints

2023-10-06 Thread Eric Blake
consent for my contribution here): https://listman.redhat.com/archives/libguestfs/2023-October/032755.html Eric Blake (2): utils: Slightly simplify human_size() info: Show human sizes for block_size values common/include/human-size.h | 14 ++ info/show.c

[Libguestfs] [libnbd PATCH 1/2] utils: Slightly simplify human_size()

2023-10-06 Thread Eric Blake
Use an array of characters instead of strings for less .data storage. Merge the loop conditional for fewer lines of code. Signed-off-by: Eric Blake --- common/include/human-size.h | 14 ++ 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/common/include/human-size.h b

[Libguestfs] [libnbd PATCH 2/2] info: Show human sizes for block_size values

2023-10-06 Thread Eric Blake
) Signed-off-by: Eric Blake --- info/show.c | 26 +++--- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/info/show.c b/info/show.c index 6aeffb54..ac483f34 100644 --- a/info/show.c +++ b/info/show.c @@ -35,6 +35,7 @@ #include "nbdinfo.h" static void sh

Re: [Libguestfs] [PATCH libnbd 3/5] common: Combine human-size.h headers into one

2023-10-06 Thread Eric Blake
i--; > +} > + } > + > + /* Set the flag to true if we're going to add a human-readable extension. > */ > + if (human) > +*human = ext[i][0] != '\0'; *human = ext[i] != '\0'; > + > + snprintf (buf, HUMAN_SIZE_LON

Re: [Libguestfs] [PATCH v7 01/12] nbd/server: Support a request payload

2023-10-05 Thread Eric Blake
On Mon, Sep 25, 2023 at 02:22:31PM -0500, Eric Blake wrote: > Upcoming additions to support NBD 64-bit effect lengths allow for the > possibility to distinguish between payload length (capped at 32M) and > effect length (64 bits, although we generally assume 63 bits because > of off_t

Re: [Libguestfs] [PATCH v7 12/12] nbd/server: Add FLAG_PAYLOAD support to CMD_BLOCK_STATUS

2023-10-05 Thread Eric Blake
l drop. > > > +payload_len = 0; > > + valid_flags |= NBD_CMD_FLAG_PAYLOAD_LEN; > > +} else { > > +request->contexts = &client->contexts; > > +} > > valid_flags |= NBD_CMD_FLAG_REQ_ONE;

Re: [Libguestfs] [PATCH v7 12/12] nbd/server: Add FLAG_PAYLOAD support to CMD_BLOCK_STATUS

2023-10-05 Thread Eric Blake
On Wed, Oct 04, 2023 at 04:55:02PM -0500, Eric Blake wrote: > > > +static int > > > +nbd_co_block_status_payload_read(NBDClient *client, NBDRequest *request, > > > + Error **errp) > > > > [..] > > > +for (i = 0

Re: [Libguestfs] [PATCH v7 12/12] nbd/server: Add FLAG_PAYLOAD support to CMD_BLOCK_STATUS

2023-10-04 Thread Eric Blake
On Sat, Sep 30, 2023 at 04:24:02PM +0300, Vladimir Sementsov-Ogievskiy wrote: > On 25.09.23 22:22, Eric Blake wrote: > > Allow a client to request a subset of negotiated meta contexts. For > > example, a client may ask to use a single connection to learn about > > both b

Re: [Libguestfs] [PATCH v7 01/12] nbd/server: Support a request payload

2023-09-28 Thread Eric Blake
On Thu, Sep 28, 2023 at 12:09:51PM +0300, Vladimir Sementsov-Ogievskiy wrote: > On 27.09.23 18:59, Eric Blake wrote: > > We could also try to be a bit more complicated by peeking at the next > > few bytes: if they look like a magic number of the next request, > > assume t

Re: [Libguestfs] [libnbd PATCH] info: Try harder for graceful disconnect from server

2023-09-27 Thread Eric Blake
On Fri, Sep 22, 2023 at 04:03:19PM -0500, Eric Blake wrote: > On Fri, Sep 22, 2023 at 09:47:55PM +0100, Richard W.M. Jones wrote: > > On Fri, Sep 22, 2023 at 12:33:36PM -0500, Eric Blake wrote: > > > There are a number of ways in which gathering information can fail. > >

Re: [Libguestfs] [PATCH v7 01/12] nbd/server: Support a request payload

2023-09-27 Thread Eric Blake
On Wed, Sep 27, 2023 at 11:55:41AM +0300, Vladimir Sementsov-Ogievskiy wrote: > On 25.09.23 22:22, Eric Blake wrote: > > Upcoming additions to support NBD 64-bit effect lengths allow for the > > possibility to distinguish between payload length (capped at 32M) and > > e

Re: [Libguestfs] LIBNBD SECURITY: Negative results from nbd_get_size() - CVE-2023-5215

2023-09-27 Thread Eric Blake
On Tue, Sep 26, 2023 at 02:12:27PM -0500, Eric Blake wrote: > We have discovered a security flaw with potential minor impact in > libnbd. > > Lifecycle > - > > Reported: 2023-09-17 Fixed: 2023-09-22 Published: 2023-09-26 > > At the time of this email, t

[Libguestfs] LIBNBD SECURITY: Negative results from nbd_get_size()

2023-09-26 Thread Eric Blake
tests, patched by Eric Blake Description --- libnbd is a Network Block Device (NBD) client library. The NBD protocol states that a server advertises the size of an export using a 64-bit unsigned number. In turn, libnbd has an API nbd_get_size() that returns int64_t, documenting that a

[Libguestfs] [PATCH v7 11/12] nbd/server: Prepare for per-request filtering of BLOCK_STATUS

2023-09-25 Thread Eric Blake
return on a per-command basis (for now, identical to the full set of negotiated contexts). Signed-off-by: Eric Blake --- v5: fix null dereference on early error [Vladimir], hoist in assertion from v4 24/24 v4: split out NBDMetaContexts refactoring to its own patch, track NBDRequests.contexts as

[Libguestfs] [PATCH v7 12/12] nbd/server: Add FLAG_PAYLOAD support to CMD_BLOCK_STATUS

2023-09-25 Thread Eric Blake
); but for NBD_OPT_GO, qemu only advertises the feature if block status is also enabled (that is, if the client does not negotiate any contexts, then NBD_CMD_BLOCK_STATUS cannot be used, so the feature is not advertised). Signed-off-by: Eric Blake --- v5: factor out 'id - NBD_MTA_ID_DIRTY_BI

[Libguestfs] [PATCH v7 08/12] nbd/client: Accept 64-bit block status chunks

2023-09-25 Thread Eric Blake
hether more than the two least-significant bits are set. Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- v5: factor out duplicate length calculation [Vladimir], add R-b v4: tweak comments and error message about count mismatch, fix setting of wide in loop [Vladimir

[Libguestfs] [PATCH v7 10/12] nbd/server: Refactor list of negotiated meta contexts

2023-09-25 Thread Eric Blake
path, dropping context in nbd_check_meta_export even when reporting an error is safer. Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- v5: rebase to master, tweak commit message [Vladimir], R-b added v4: new patch split out from v3 13/14, with smaller impact (quit trying

[Libguestfs] [PATCH v7 09/12] nbd/client: Request extended headers during negotiation

2023-09-25 Thread Eric Blake
t. Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- v5: add R-b v4: rebase to earlier changes, tweak commit message for why qemu-nbd connection to /dev/nbd cannot use extended mode [Vladimir] --- nbd/client-connection.c | 2 +- nb

[Libguestfs] [PATCH v7 07/12] nbd/client: Initial support for extended headers

2023-09-25 Thread Eric Blake
insist that the server is compliant. Signed-off-by: Eric Blake --- v5: fix logic bug on error reporting [Vladimir] v4: split off errp handling to separate patch [Vladimir], better function naming [Vladimir] --- include/block/nbd.h | 3 +- block/nbd.c | 2 +- nbd/client.c| 104

[Libguestfs] [PATCH v7 05/12] nbd/server: Enable initial support for extended headers

2023-09-25 Thread Eric Blake
lient since qemu only requests the single context base:allocation. Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- v5: add R-b, s/8.1/8.2/ v4: split out parts into earlier patches, rebase to earlier changes, simplify handling of generic replies, retitle (compare to v3

[Libguestfs] [PATCH v7 04/12] nbd/server: Support 64-bit block status

2023-09-25 Thread Eric Blake
ecause all lengths are coming from the block layer, and we know that the block layer does not support images larger than off_t (if lengths were coming from the network, the story would be different). Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- v5: stronger justificat

[Libguestfs] [PATCH v7 03/12] nbd/server: Prepare to send extended header replies

2023-09-25 Thread Eric Blake
Although extended mode is not yet enabled, once we do turn it on, we need to reply with extended headers to all messages. Update the low level entry points necessary so that all other callers automatically get the right header based on the current mode. Signed-off-by: Eric Blake Reviewed-by

[Libguestfs] [PATCH v7 06/12] nbd/client: Plumb errp through nbd_receive_replies

2023-09-25 Thread Eric Blake
Instead of ignoring the low-level error just to refabricate our own message to pass to the caller, we can just plumb the caller's errp down to the low level. Signed-off-by: Eric Blake --- v5: set errp on more failure cases [Vladimir], typo fix v4: new patch [Vladimir] --- block/nbd.c

[Libguestfs] [PATCH v7 01/12] nbd/server: Support a request payload

2023-09-25 Thread Eric Blake
security risk). Note that we do not support the payload version of BLOCK_STATUS yet. Signed-off-by: Eric Blake --- v7: another try at better logic [Vladimir] v5: retitled from v4 13/24, rewrite on top of previous patch's switch statement [Vladimir] v4: less indentation on several 'if&#

[Libguestfs] [PATCH v7 02/12] nbd/server: Prepare to receive extended header requests

2023-09-25 Thread Eric Blake
that stops talking to us partway through a packet and thus not worth coding around. Signed-off-by: Eric Blake Reviewed-by: Vladimir Sementsov-Ogievskiy --- v6: fix sign extension bug v5: no change v4: new patch, split out from v3 9/14 --- nbd/nbd-internal.h | 5 - nbd/server.c

[Libguestfs] [PATCH v7 00/12] NBD 64-bit extensions for qemu

2023-09-25 Thread Eric Blake
v6 was here: https://lists.gnu.org/archive/html/qemu-devel/2023-08/msg05231.html Since then: - patches v6 1-5 included in pull request - patch v6 6 logic improved, now v7 patch 1 - rebased to master Still needing review: - patch 1,6,7,11,12 Eric Blake (12): nbd/server: Support a request

Re: [Libguestfs] [EXTERNAL] - Re: Libguestfs desired version installation

2023-09-25 Thread Eric Blake
t; will make any difference. your distro support contract will likewise expect you to put in more effort of describing the actual scenario you are trying to solve, rather than the secondary problem of "can I upgrade libguestfs" that may or may not address the primary problem th

Re: [Libguestfs] [libnbd PATCH] info: Try harder for graceful disconnect from server

2023-09-22 Thread Eric Blake
On Fri, Sep 22, 2023 at 09:47:55PM +0100, Richard W.M. Jones wrote: > On Fri, Sep 22, 2023 at 12:33:36PM -0500, Eric Blake wrote: > > There are a number of ways in which gathering information can fail. > > But when possible, it makes sense to let the server know that we are > >

[Libguestfs] [libnbd PATCH] info: Try harder for graceful disconnect from server

2023-09-22 Thread Eric Blake
/home/eblake/libnbd/info/.libs/nbdinfo: nbd_get_size: server claims size 9223372036854781440 which does not fit in signed result: Value too large for defined data type if nbdinfo is run in the same terminal as qemu-nbd. Signed-off-by: Eric Blake --- info/main.c | 13 + 1 file cha

Re: [Libguestfs] [libnbd PATCH v2 0/6] Fix fuzzer fallout

2023-09-22 Thread Eric Blake
On Thu, Sep 21, 2023 at 03:57:59PM -0500, Eric Blake wrote: > Cat's out of the bag: Rich's fuzzer run found not one, but two > independent assertion failures that a malicious server could trigger > in my recent 64-bit extension code additions. What's more, in the > pr

[Libguestfs] nbdinfo default output [was: [libnbd PATCH v2 6/6] info: Tolerate missing size]

2023-09-22 Thread Eric Blake
On Fri, Sep 22, 2023 at 09:03:53AM -0500, Eric Blake wrote: > > > $ ./run nbdinfo nbd://localhost > > > protocol: newstyle-fixed without TLS, using extended packets > > > ... > > > block_size_maximum: 33554432 For the human-readable output, should

Re: [Libguestfs] [libnbd PATCH v2 6/6] info: Tolerate missing size

2023-09-22 Thread Eric Blake
On Fri, Sep 22, 2023 at 02:19:32PM +0200, Laszlo Ersek wrote: > On 9/21/23 22:58, Eric Blake wrote: > > As previous patches showed, the NBD spec does not yet forbid a server > > sending us a size that does not fit in int64_t. We should gracefully > > handle this during nbdinf

[Libguestfs] [libnbd PATCH v2 3/6] api: Sanitize sizes larger than INT64_MAX

2023-09-21 Thread Eric Blake
here offsets are ignored (either a read-only export of indefinite length, or an append-only export data sink - either way, basically turning NBD into a cross-system FIFO rather than a seekable device); if such an extension materializes, we'd probably add a named constant negative sentinel value

[Libguestfs] [libnbd PATCH v2 0/6] Fix fuzzer fallout

2023-09-21 Thread Eric Blake
sitive export size should probably favor nbd_get_size()<0 over nbd_get_size()==-1). Eric Blake (6): states: Tweak comment in OPT_GO state handler fuzzing: Disable client-side strictness checks api: Sanitize sizes larger than INT64_MAX block_status: Fix assertion with large server size block_

[Libguestfs] [libnbd PATCH v2 5/6] block_status: Fix assertion on bad 64-bit block status reply

2023-09-21 Thread Eric Blake
ased libnbd. Thus, there is no reason to create a CVE, although since I will already be doing a security info email about previous patches also addressing fuzzer findings, I can mention this at the same time. Fixes: ab992766cd ("block_status: Accept 64-bit extents during block status") Tha

[Libguestfs] [libnbd PATCH v2 1/6] states: Tweak comment in OPT_GO state handler

2023-09-21 Thread Eric Blake
. Signed-off-by: Eric Blake Reviewed-by: Richard W.M. Jones Reviewed-by: Laszlo Ersek --- generator/states-newstyle-opt-go.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/generator/states-newstyle-opt-go.c b/generator/states-newstyle-opt-go.c index 2ef440d4..5bc9a9ae

[Libguestfs] [libnbd PATCH v2 2/6] fuzzing: Disable client-side strictness checks

2023-09-21 Thread Eric Blake
records, where each record is the API to call and then the server's response; right now, the sequence of APIs called is hard-coded, which is not as powerful at testing potential cross-command coupling. But that's a project for another day.] Signed-off-by: Eric Blake Reviewed-by: Richard

[Libguestfs] [libnbd PATCH v2 4/6] block_status: Fix assertion with large server size

2023-09-21 Thread Eric Blake
able releases. Fixes: e8d837d306 ("block_status: Add some sanity checking of server lengths", v1.17.4) Thanks: Richard W.M. Jones Signed-off-by: Eric Blake --- generator/states-reply-chunk.c | 49 ++ generator/C.ml | 2 +- 2 files chang

[Libguestfs] [libnbd PATCH v2 6/6] info: Tolerate missing size

2023-09-21 Thread Eric Blake
"block_size_maximum": 33554432, "export-size-str": "unavailable" } ] } Sadly, since writing a server with such large export sizes requires a one-off hack, I don't see the point in adding a unit test. Signed-off-by: Eric Blake --- info/show.c | 25 +

[Libguestfs] [libnbd PATCH] generator: Fix assertion with ill-formed 64-bit block status reply

2023-09-18 Thread Eric Blake
tweak to pending qemu patches that could trigger it. Not creating a CVE as it only appears in unstable releases. Fixes: ab992766cd ("block_status: Accept 64-bit extents during block status") Thanks: Richard W.M. Jones Signed-off-by: Eric Blake --- I'm investigating another crash

Re: [Libguestfs] Plans for nbdkit 1.36 & libnbd 1.18

2023-09-13 Thread Eric Blake
elease. > > I'll prepare draft release notes in the coming weeks. > > Rich. > -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs

Re: [Libguestfs] [PATCH nbdkit 10/10] XXX docs: Remove references to -U - when it is implicit

2023-09-11 Thread Eric Blake
e, and therefore which could be hoisted into 9/10 if desired. But I see no harm in leaving it here. Obviously, we'd better not forget to apply this at the correct later date, and you'll have to touch up the commit message at that point. But once we are ready, feel free to ad

Re: [Libguestfs] [PATCH nbdkit 09/10] tests: Remove references to -U - when it is implicit

2023-09-11 Thread Eric Blake
10/10 (where changing them now before the stable release is imminent could cause confusion). Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org ___ Libguestfs mailing list Libguest

Re: [Libguestfs] [PATCH nbdkit 08/10] tests: Be punctilious about using requires_run in tests that use --run

2023-09-11 Thread Eric Blake
rust/test-ramdisk.sh | 2 ++ > tests/test-S3.sh | 1 + > tests/test-vsock.sh | 1 + > 75 files changed, 76 insertions(+) Big, but mechanical and makes sense. Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer

Re: [Libguestfs] [PATCH nbdkit 07/10] tests/test-parallel-*.sh: Remove redundant comment

2023-09-11 Thread Eric Blake
On Sat, Sep 09, 2023 at 02:57:55PM +0100, Richard W.M. Jones wrote: > --- > tests/test-parallel-file.sh | 1 - > tests/test-parallel-nbd.sh | 1 - > tests/test-parallel-sh.sh | 1 - > 3 files changed, 3 deletions(-) Reviewed-by: Eric Blake -- Eric Blake, Principal Software E

Re: [Libguestfs] [PATCH nbdkit 06/10] server: Make --run imply -U -

2023-09-11 Thread Eric Blake
ea, although > we use it in one test.) > --- > docs/nbdkit-captive.pod | 7 --- > docs/nbdkit.pod | 9 - > server/main.c | 9 + > 3 files changed, 17 insertions(+), 8 deletions(-) > Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer

Re: [Libguestfs] [PATCH nbdkit 04/10] server: Calculate $uri in one place

2023-09-11 Thread Eric Blake
7; instead of \, > > may be different, but the resulting string as parsed by the shell is > > the same). My point was that pre-patch, we had: nbd+unix://\?socket=... and post-patch, we have: 'nbd+unix://?socket=...' but both forms are equally quoted; after the shell removes

Re: [Libguestfs] [PATCH nbdkit 05/10] server: Add the NBD URI to debug output

2023-09-11 Thread Eric Blake
c > @@ -637,6 +637,8 @@ main (int argc, char *argv[]) > * Note this may be NULL. > */ > uri = make_uri (); > + if (uri) > +debug ("NBD URI: %s", uri); Reviewed-by: Eric Blake Do we also want to output a debug statement when a URI is not p

Re: [Libguestfs] [PATCH nbdkit 04/10] server: Calculate $uri in one place

2023-09-11 Thread Eric Blake
', fp); > + fputs (port, fp); > +} > +if (export_name && strcmp (export_name, "") != 0) { > + putc ('/', fp); > + uri_quote (export_name, fp); > +} > +break; > + case SERVICE_MODE_TCPIP: > + fprintf (fp, &qu

Re: [Libguestfs] [PATCH nbdkit 03/10] server: Don't set port as a side effect

2023-09-11 Thread Eric Blake
On Sat, Sep 09, 2023 at 02:57:51PM +0100, Richard W.M. Jones wrote: > This removes another long-range code dependency. > --- > server/sockets.c | 10 +- > 1 file changed, 5 insertions(+), 5 deletions(-) > Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engine

Re: [Libguestfs] [PATCH nbdkit 02/10] server: Don't set export_name as a side effect of using --run

2023-09-11 Thread Eric Blake
me will be NULL. > --- > server/captive.c | 12 +--- > 1 file changed, 5 insertions(+), 7 deletions(-) Reviewed-by: Eric Blake For other reviewers, remember that -e only impacts $uri during --run (it is still up to the plugin whether the client must connect to a particular expor

Re: [Libguestfs] [PATCH nbdkit 01/10] server: Introduce service_mode concept

2023-09-11 Thread Eric Blake
data > uri: nbd+vsock://1:10809/ > ... Nice. > --- > server/internal.h | 11 +++ > server/captive.c | 56 --- > server/main.c | 75 ++++++- > 3 files changed, 91 insertions(+), 51 de

Re: [Libguestfs] [PATCH nbdkit 00/10] Make --run imply -U -

2023-09-11 Thread Eric Blake
On Mon, Sep 11, 2023 at 01:11:46PM +0100, Richard W.M. Jones wrote: > On Mon, Sep 11, 2023 at 07:08:48AM -0500, Eric Blake wrote: > > On Sat, Sep 09, 2023 at 02:57:48PM +0100, Richard W.M. Jones wrote: > > > Should have done this a long time ago. I feel it is about time w

Re: [Libguestfs] [PATCH nbdkit 00/10] Make --run imply -U -

2023-09-11 Thread Eric Blake
still installed on various CI platforms). > > Last patch updating the documentation wouldn't be applied any time > soon, so that the old docs stay around on the website. Also a good idea. -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualiza

Re: [Libguestfs] [libnbd PATCH v4 25/25] api: Add LIBNBD_STRICT_AUTO_FLAG control to nbd_set_strict

2023-09-08 Thread Eric Blake
On Wed, Aug 02, 2023 at 08:50:45PM -0500, Eric Blake wrote: > We've previously documented that nbd_set_strict() can add new bits, > defaulting to on to provide client-side safety, but which can be > overridden when performing specific integration tests against a > serv

Re: [Libguestfs] libnbd | Failed pipeline for master | 0b587f8a

2023-09-07 Thread Eric Blake
.com/nbdkit/libnbd/-/commit/0b587f8a4f3e61e27985b8a3f66a3d7da45c00ef > ) > Commit Message: info: Prefer NBD_OPT_INFO when possible > > Now th... > Commit Author: Eric Blake ( https://gitlab.com/ebblake ) > > > Pipeline #996353812 ( https://gitlab.com/nbdkit/libnbd/-/pipel

Re: [Libguestfs] [libnbd PATCH] info: Prefer NBD_OPT_INFO when possible

2023-09-07 Thread Eric Blake
On Tue, Sep 05, 2023 at 03:23:58PM +0100, Richard W.M. Jones wrote: > On Tue, Sep 05, 2023 at 08:47:34AM -0500, Eric Blake wrote: > > diff --git a/info/info-list-uris.sh b/info/info-list-uris.sh > > index 0d6a16a0..d8ea9108 100755 > > --- a/info/info-list-uris.sh > > +

Re: [Libguestfs] [libnbd PATCH] rust: Use mio::poll instead of requiring epoll

2023-09-07 Thread Eric Blake
On Wed, Sep 06, 2023 at 05:02:06PM -0500, Eric Blake wrote: > > Okay, I'll merge in that aspect of my original along with your other > > improvements, and push something soon. Fingers crossed that we'll > > finally get a green CI run today. > > Now in as 759798

Re: [Libguestfs] [libnbd PATCH] rust: Use mio::poll instead of requiring epoll

2023-09-06 Thread Eric Blake
On Tue, Sep 05, 2023 at 09:45:44AM -0500, Eric Blake wrote: > On Mon, Sep 04, 2023 at 06:59:15PM +, Tage Johansson wrote: > > > > > > +// can be both readable and writable, but we survive just fine > > > > +// if we only see one direc

Re: [Libguestfs] [PATCH nbdkit] server: Move size parsing code (nbdkit_parse_size) to common/include

2023-09-05 Thread Eric Blake
_t or int64_t; even though size_t is such a naturally occuring type, the fact that it is not uniform width makes it trickier to work with). > > > + > > > + if (INT64_MAX / scale < size) { > > > +*error = "could no

Re: [Libguestfs] [libnbd PATCH] rust: Use mio::poll instead of requiring epoll

2023-09-05 Thread Eric Blake
g > > this will return Ok(()). However we’re not guaranteeing this behaviour > > as this depends on the OS. > > So I guess it is best to ignore any errors from the poll call as in your > patch. Okay, I'll merge in that aspect of my original along with your other improvement

Re: [Libguestfs] [libnbd PATCH v9 6/7] rust: async: Add an example

2023-09-05 Thread Eric Blake
On Mon, Sep 04, 2023 at 04:56:07PM +, Tage Johansson wrote: > > On 9/1/2023 10:41 PM, Eric Blake wrote: > > On Sat, Aug 26, 2023 at 11:29:59AM +, Tage Johansson wrote: > > > This patch adds an example using the asynchronous Rust bindings. > > &g

Re: [Libguestfs] [libnbd PATCH 0/3] Simplify nbd_shutdown vs. opt mode

2023-09-05 Thread Eric Blake
On Fri, Sep 01, 2023 at 10:28:52PM +0100, Richard W.M. Jones wrote: > On Tue, Aug 29, 2023 at 05:20:40PM -0500, Eric Blake wrote: > > While working on a larger set of patches to make nbdinfo favor > > NBD_OPT_INFO over NBD_OPT_GO where possible (which requires use of > > nbd_s

[Libguestfs] [libnbd PATCH] info: Prefer NBD_OPT_INFO when possible

2023-09-05 Thread Eric Blake
t a double-free bug in 'nbdkit file dir=...' when querying opt_info on the default name, so a couple of unit tests need to avoid false negatives on platforms where nbdkit commit 39d62de9 is not yet backported. Signed-off-by: Eric Blake --- info/info-list-uris.sh | 9 - in

Re: [Libguestfs] [libnbd PATCH] rust: Use mio::poll instead of requiring epoll

2023-09-04 Thread Eric Blake
On Mon, Sep 04, 2023 at 04:07:54PM +, Tage Johansson wrote: > From: Eric Blake > > CI shows our async handle fails to build on FreeBSD and MacOS (where > epoll() is not available as a syscall, and therefore not available as > a Rust crate). We can instead accomplish the sam

Re: [Libguestfs] [libnbd PATCH v9 6/7] rust: async: Add an example

2023-09-01 Thread Eric Blake
or from a comment that has gone stale after refactoring in the original C code. > + > +let mut buf = [0u8; BUFFER_SIZE]; > +let nbd = nbd.clone(); > +if rng.gen() { > + join_set.spawn(async move { > +nbd.pread(&mut b

Re: [Libguestfs] [libnbd PATCH 2/2] rust: Use mio::poll instead of requiring epoll

2023-09-01 Thread Eric Blake
On Fri, Sep 01, 2023 at 08:35:58AM +, Tage Johansson wrote: > > On 8/30/2023 10:11 PM, Eric Blake wrote: > > CI shows our async handle fails to build on FreeBSD and MacOS (where > > epoll() is not available as a syscall, and therefore not available as > > a Rust

Re: [Libguestfs] [nbdkit PATCH] sh: Allow pwrite to not consume all data

2023-08-31 Thread Eric Blake
ugins as a data sink still have to inject 'cat >/dev/null' to drain stdin, in order to pass under heavy load when EPIPE even happens in the first place (I was surprised at how hard it is to lose the race and actually get an EPIPE message from nbdkit talking to the sh plugin script on my machine). -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs

Re: [Libguestfs] [nbdkit PATCH] sh: Allow pwrite to not consume all data

2023-08-31 Thread Eric Blake
On Thu, Aug 31, 2023 at 11:12:59AM +0200, Laszlo Ersek wrote: > On 8/31/23 10:02, Richard W.M. Jones wrote: > > > > On Wed, Aug 30, 2023 at 05:21:19PM -0500, Eric Blake wrote: > >> I hit another transient failure in libnbd CI when a poorly-written > >> eval sc

Re: [Libguestfs] [PATCH nbdkit] sh: In pwrite, allow scripts to ignore stdin

2023-08-31 Thread Eric Blake
xit 2 ;; > +esac > +EOF > + > +nbdsh -u "nbd+unix://?socket=$sock" -c ' > +buf = bytearray(16*1024*1024) > +h.pwrite(buf, 0) > + > +try: > +h.pwrite(buf, 32*1024*1024) > +assert False > +except nbd.Error: > +# Expect an error here.

[Libguestfs] [nbdkit PATCH] sh: Allow pwrite to not consume all data

2023-08-30 Thread Eric Blake
immediate return of EIO to the client. Signed-off-by: Eric Blake --- I probably need to add unit test coverage of this before committing (although proving that I win the data race on a client process exiting faster than the parent can write enough data to hit EPIPE is hard). plugins/sh/nbdkit-sh

Re: [Libguestfs] [libnbd PATCH v9 5/7] rust: async: Add a couple of integration tests

2023-08-30 Thread Eric Blake
On Wed, Aug 30, 2023 at 12:16:07PM -0500, Eric Blake wrote: > > error[E0425]: cannot find value `EPOLL_CTL_ADD` in crate `libc` > > https://www.reddit.com/r/rust/comments/65kflg/does_rust_have_native_epoll_support/ > mentions how epoll is Linux-specific, and has comments about to

[Libguestfs] [libnbd PATCH 1/2] maint: Favor 4-space indent in .rs files

2023-08-30 Thread Eric Blake
Since rustfmt favors 4-space indent on .rs files, we should do likewise to reduce the churn when running rustfmt after editing in an emacs session set to honor editorconfig. Signed-off-by: Eric Blake --- .editorconfig | 4 1 file changed, 4 insertions(+) diff --git a/.editorconfig b

[Libguestfs] [libnbd PATCH 2/2] rust: Use mio::poll instead of requiring epoll

2023-08-30 Thread Eric Blake
, and kqueue on BSD). Fixes: 223a9965 ("rust: async: Create an async friendly handle type") CC: Tage Johansson Signed-off-by: Eric Blake --- rust/Cargo.toml | 2 +- rust/src/async_handle.rs | 46 +--- 2 files changed, 30 insertions(+), 18

[Libguestfs] [libnbd PATCH 0/2] (Attempt to) fix Rust on BSD-based builds

2023-08-30 Thread Eric Blake
review on patch 2 to see if my Rust makes sense. Eric Blake (2): maint: Favor 4-space indent in .rs files rust: Use mio::poll instead of requiring epoll rust/Cargo.toml | 2 +- rust/src/async_handle.rs | 46 +--- .editorconfig| 4

Re: [Libguestfs] [libnbd PATCH v9 5/7] rust: async: Add a couple of integration tests

2023-08-30 Thread Eric Blake
On Wed, Aug 30, 2023 at 08:42:00AM -0500, Eric Blake wrote: > On Tue, Aug 29, 2023 at 05:17:19PM -0500, Eric Blake wrote: > > On Sat, Aug 26, 2023 at 11:29:58AM +, Tage Johansson wrote: > > > Add a couple of integration tests as rust/tests/test_async_*.rs. They > > &

Re: [Libguestfs] [libnbd PATCH v9 5/7] rust: async: Add a couple of integration tests

2023-08-30 Thread Eric Blake
On Tue, Aug 29, 2023 at 05:17:19PM -0500, Eric Blake wrote: > On Sat, Aug 26, 2023 at 11:29:58AM +, Tage Johansson wrote: > > Add a couple of integration tests as rust/tests/test_async_*.rs. They > > are very similar to the tests for the synchronous API. > > -

[Libguestfs] [libnbd PATCH 3/3] info: Simplify shutdown calls

2023-08-29 Thread Eric Blake
nbd_opt_abort or nbd_shutdown that would not fail; but now that nbd_shutdown has a new flag, it is simpler to not have to worry about it. Signed-off-by: Eric Blake --- info/list.c | 8 +--- info/main.c | 4 +--- 2 files changed, 2 insertions(+), 10 deletions(-) diff --git a/info/list.c b/info/list.c

[Libguestfs] [libnbd PATCH 2/3] api: Add new COVER_OPT_MODE flag to nbd_shutdown

2023-08-29 Thread Eric Blake
test to show the effect of the new flag, and the next commit will simplify nbdinfo by utilizing it. Signed-off-by: Eric Blake --- generator/API.ml | 21 - lib/disconnect.c | 15 +++ tests/shutdown-opt-mode.c | 27 ++- 3

[Libguestfs] [libnbd PATCH 1/3] tests: Test behavior of nbd_shutdown during opt mode

2023-08-29 Thread Eric Blake
decide between nbd_opt_abort or nbd_shutdown (if they call both, one of the two will give an error message about being used from the wrong state). Add some unit test coverage of this prior to enhancing the API to make a clean shutdown easier for clients. Signed-off-by: Eric Blake --- tests

[Libguestfs] [libnbd PATCH 0/3] Simplify nbd_shutdown vs. opt mode

2023-08-29 Thread Eric Blake
nbd_shutdown has the desired capability (if we allow it to auto-opt_abort without a flag, it's harder to tell whether we are running against an older libnbd where it errors out instead). Eric Blake (3): tests: Test behavior of nbd_shutdown during opt mode api: Add new COVER_OPT_MODE fl

Re: [Libguestfs] [libnbd PATCH v9 5/7] rust: async: Add a couple of integration tests

2023-08-29 Thread Eric Blake
copy/pasted the synchronous 460 into 465, I'll probably end up doing the same for the async version, unless you beat me to it. -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org ___ Libguestfs mai

Re: [Libguestfs] [libnbd PATCH 07/10] rust: Add a couple of integration tests

2023-08-29 Thread Eric Blake
list, while the Rust code does not. You can see the same thing in the Python tests (calling opt_abort before closing the handle). This means that for the Rust tests, the server is more likely to issue an error message about the client abruptly disconnecting, which doesn't reall

Re: [Libguestfs] [libnbd PATCH v4 20/25] generator: Actually request extended headers

2023-08-28 Thread Eric Blake
On Wed, Aug 02, 2023 at 08:50:40PM -0500, Eric Blake wrote: > This is the culmination of the previous patches' preparation work for > using extended headers when possible. The new states in the state > machine are copied extensively from our handling of > OPT_STRUCTURED_REPLY

Re: [Libguestfs] [libnbd PATCH v8 06/10] rust: async: Create an async friendly handle type

2023-08-24 Thread Eric Blake
imeout to 0 means that it will return immediately. > +epoll::wait(epfd, 0, std::slice::from_mut(&mut revent))?; > +let revents = Events::from_bits(revent.events).unwrap(); > +if !revents.contains(Events::EPOLLIN) { > +ready_guard.clear_read

Re: [Libguestfs] [libnbd PATCH v8 05/10] generator: Add information about asynchronous handle calls

2023-08-24 Thread Eric Blake
connected", false). > *) > +| ChangesState of string * bool > and link = > | Link of string (** link to L *) > | SectionLink of string(** link to L *) > -- > 2.41.0 Grammar changes can be made while merging; no need to respin unless later patches require more subs

Re: [Libguestfs] [libnbd PATCH v8 04/10] rust: Make it possible to run examples with a URI

2023-08-22 Thread Eric Blake
On Tue, Aug 22, 2023 at 04:45:40PM -0500, Eric Blake wrote: > For 1-4, > Reviewed-by: Eric Blake > > and I've gone ahead and pushed them (55fec56c..e5563c0d), in order to > see the effect on CI. Sorry, wrong commit ids (those were on a local tree before rebasing); actual

Re: [Libguestfs] [libnbd PATCH v4 01/25] block_status: Add some sanity checking of server lengths

2023-08-22 Thread Eric Blake
On Fri, Aug 04, 2023 at 11:49:18AM +0100, Richard W.M. Jones wrote: > On Wed, Aug 02, 2023 at 08:50:21PM -0500, Eric Blake wrote: > > Previously, we had not been doing any validation of server extent > > responses, which means a client query at an offset near the end of the > &

Re: [Libguestfs] [libnbd PATCH v8 04/10] rust: Make it possible to run examples with a URI

2023-08-22 Thread Eric Blake
;' to set a good example about how to avoid rare file name expansion interference. I made that change in place, as well as tweaking ocaml/examples/get_size.ml where you copied from. For 1-4, Reviewed-by: Eric Blake and I've gone ahead and pushed them (55fec56c..e5563c0d), in order t

Re: [Libguestfs] [nbdkit PATCH] cc: Allow configuration without absolute paths

2023-08-18 Thread Eric Blake
On Fri, Aug 18, 2023 at 08:27:45AM -0500, Eric Blake wrote: > In https://gitlab.com/nbdkit/nbdkit/-/merge_requests/30, Khem reports > that in a cross-compilation environment, nbdkit embeds the absolute > name of the cross-compiler into the resulting cc plugin, even though > runnin

[Libguestfs] [nbdkit PATCH] cc: Allow configuration without absolute paths

2023-08-18 Thread Eric Blake
the build environment variable as suggested in that merge request, I found it cleaner to just add additional precious variables to be set at configure time, as in: ./configure CC=/path/to/cross-compiler CC_PLUGIN_CC='ccache gcc' ... Reported-by: Khem Raj Signed-off-by: Eric Blake --- g

Re: [Libguestfs] [libnbd PATCH] golang: Bump minimum Go version to 1.17

2023-08-17 Thread Eric Blake
er if 'go run .' fails if 'go mod tidy' is still allowed to succeed right after. With that fixed, I got a few more green lines in the CI (before turning back to a bunch of red now that I added Rust into the CI...) -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs

Re: [Libguestfs] [libnbd PATCH] ci: Update lcitool to request cargo during CI

2023-08-17 Thread Eric Blake
On Tue, Aug 15, 2023 at 04:33:38PM -0500, Eric Blake wrote: > Regenerate the CI build scripts with: > > ../libvirt-ci/bin/lcitool manifest ci/manifest.yml > > using libvirt-ci bumped with a pending patch: > https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/424 >

Re: [Libguestfs] [libnbd PATCH v7 0/9] Rust Bindings for Libnbd

2023-08-17 Thread Eric Blake
On Thu, Aug 17, 2023 at 04:13:36AM +, Tage Johansson wrote: > > On 8/16/2023 9:11 PM, Eric Blake wrote: > > On Thu, Aug 10, 2023 at 11:24:27AM +, Tage Johansson wrote: > > > This is the 7th version of the Rust bindings for Libnbd. It is more or > > > les

Re: [Libguestfs] [libnbd PATCH v7 3/9] generator: Add information about the lifetime of closures

2023-08-16 Thread Eric Blake
he closure may be used any number of times. > + (E.g., list callback.) *) > and enum = { >enum_prefix : string;(** prefix of each enum variant *) >enums : (string * int) list (** enum names and their values in C *) > -- > 2.41.0 > >

Re: [Libguestfs] [libnbd PATCH v5 02/12] rust: Add a couple of integration tests

2023-08-16 Thread Eric Blake
hunks here may grow over time as more features get // automatically negotiated, so merely check that they are non-zero. -nbd.connect_command(&[ - "nbdkit", -"-s", -"--exit-with-parent", -

  1   2   3   4   5   6   7   8   9   10   >