Re: [Libguestfs] [Qemu-devel] Why is virt-resize designed to involve two disks?

2014-09-23 Thread Eric Blake
es the fact that virt-resize is doing MUCH more than resizing the block device - it is also copying entire filesystems from the old location to the new resized location for optimal initial placement. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library ht

Re: [Libguestfs] Point-in-time snapshots

2015-03-27 Thread Eric Blake
tion, it shouldn't matter if the drive-backup failed to send all guest sectors modified after the point in time, so long as every sector you read is accurate (either because the guest hasn't touched it in the meantime; or because even though the guest touched it after the point in time, y

Re: [Libguestfs] Point-in-time snapshots

2015-03-27 Thread Eric Blake
optimizing out the writes to the NBD target for sectors you don't care about, and doesn't quite address the desire for making random reads take priority over linear streaming of dirty blocks, but it might help. -- Eric Blake eblake redhat com

Re: [Libguestfs] [libvirt] Support for qemu snapshot=on drives in libvirt

2012-10-05 Thread Eric Blake
mu-img. As for why qcow2 creation is slow, I don't know what we can do about it. > > Any thoughts on this before I implement something ... We definitely need to tie it into the XML that has already been designated for this purpose: . -- Eric Blake ebl...@redhat.com+1-919-301-32

Re: [Libguestfs] [libvirt] Support for qemu snapshot=on drives in libvirt

2012-10-05 Thread Eric Blake
On 10/05/2012 06:29 AM, Eric Blake wrote: > On 10/05/2012 05:53 AM, Richard W.M. Jones wrote: >> >> And looking even more closely, I see disk->transient (not >> implemented in the qemu driver ... why?) > > Lack of developer time :) [My usual excuse?

Re: [Libguestfs] [libvirt] Support for qemu snapshot=on drives in libvirt

2012-10-05 Thread Eric Blake
On 10/05/2012 05:53 AM, Richard W.M. Jones wrote: > > And looking even more closely, I see disk->transient (not > implemented in the qemu driver ... why?) Lack of developer time :) [My usual excuse?] -- Eric Blake ebl...@redhat.com+1-919-301-3266 Libvirt virtualization

Re: [Libguestfs] [libvirt-users] Resize errors with virt-resize/vgchange

2013-07-22 Thread Eric Blake
> > The image is the 222G /home partition only. The other partitions are > on another image. > > Any ideas how to troubleshoot this would be greatly appreciated. > Thanks, > Alex > > ___ > libvirt-users mailing list &

Re: [Libguestfs] Request to relicense hash gnulib module to LGPLv2+

2013-09-13 Thread Eric Blake
rather than our current policy of tracking down all contributors and asking them to use their grant-back clause of their FSF copyright assignment as our backdoor of not having to involve the FSF. -- Eric Blake eblake redhat com+1-919-301-3266 Libvirt virtualization library http://libvi

[Libguestfs] [nbdkit PATCH 0/2] retry: add support for retrying .open

2023-01-27 Thread Eric Blake
implementation; but in the meantime, my work to allow filters to open independent backends has made it feasible to implement. Eric Blake (2): retry: Add in retry support during .open retry: Test previous patch tests/Makefile.am| 2 + server/backend.c | 7 ++- filters/retry

[Libguestfs] [nbdkit PATCH 1/2] retry: Add in retry support during .open

2023-01-27 Thread Eric Blake
Now that a filter can open a backend as many times as it wants, there's no longer a technical reason we can't retry .open. However, adding retry logic here does mean we have to weaken an assert in the server backend code, since prepare can now be reached more than once. Test coverage will be adde

[Libguestfs] [nbdkit PATCH 2/2] retry: Test previous patch

2023-01-27 Thread Eric Blake
Separate commit to allow rearranging patches to demonstrate that open can now be retried. --- tests/Makefile.am| 2 + tests/test-retry-open.sh | 85 2 files changed, 87 insertions(+) create mode 100755 tests/test-retry-open.sh diff --git a/tests/

Re: [Libguestfs] [nbdkit PATCH 1/2] retry: Add in retry support during .open

2023-01-27 Thread Eric Blake
On Fri, Jan 27, 2023 at 02:41:22PM -0600, Eric Blake wrote: > Now that a filter can open a backend as many times as it wants, > there's no longer a technical reason we can't retry .open. However, > adding retry logic here does mean we have to weaken an assert in the > serv

Re: [Libguestfs] [nbdkit PATCH 1/2] retry: Add in retry support during .open

2023-01-31 Thread Eric Blake
On Sat, Jan 28, 2023 at 01:47:32PM +, Richard W.M. Jones wrote: > On Fri, Jan 27, 2023 at 02:41:22PM -0600, Eric Blake wrote: > > Now that a filter can open a backend as many times as it wants, > > there's no longer a technical reason we can't retry .open. However, &g

Re: [Libguestfs] [PATCH libnbd v2 3/4] generator: Add APIs to get/set the socket activation socket name

2023-01-31 Thread Eric Blake
the name "stored" assigned. Note that, if multiple file descriptors are submitted at once, the specified name will be assigned to all of them. In order to assign different names to submitted file descriptors, submit them in separate invocations of sd_pid_notify_with_fds(). The nam

Re: [Libguestfs] [PATCH libnbd v2 4/4] generator/states-connect-socket-activation.c: Set LISTEN_FDNAMES

2023-01-31 Thread Eric Blake
ies or by my idea of an alternative API that takes the socket name alongside the argv; and whether we keep to our 32-byte [[:alnum:]] limit or instead allow for a larger name up to 255 bytes in the regex range notation by ASCII byte value [\x20-\x39\x41-\x7e] (aka [ -9;-~], or [^\x00-\x1f\x7f-\xff]

Re: [Libguestfs] [libnbd PATCH v2 3/3] nbdsh: Improve --help and initial banner contents.

2023-01-31 Thread Eric Blake
On Tue, Jan 31, 2023 at 11:52:27AM +, Richard W.M. Jones wrote: > On Tue, Jan 31, 2023 at 01:33:25PM +0200, Nir Soffer wrote: > > On Tue, Jan 31, 2023 at 12:34 AM Richard W.M. Jones > > wrote: > > > > > > On Fri, Nov 04, 2022 at 04:18:31PM -0500, Eric

Re: [Libguestfs] [PATCH libnbd] generator: Pass LISTEN_FDNAMES=nbd with systemd socket activation

2023-01-31 Thread Eric Blake
gt; > > > execvp() scans PATH, and is not safe to use in this concept. > > That's quite annoying. > > > I think we should call execve() instead. First, it is async-signal-safe. > > Second, it could take "env.ptr" directly; I do find the "environ" &g

Re: [Libguestfs] [PATCH libnbd v2 3/4] generator: Add APIs to get/set the socket activation socket name

2023-01-31 Thread Eric Blake
You have a strong point there. Just because systemd allows it doesn't make it wise; I'm a big fan of "it's easier to resrict now and loosen later when we see the need", as being easier than "let's try and be as loose as we can now, then regre

Re: [Libguestfs] Preliminary release notes for libguestfs 1.50, guestfs-tools 1.50

2023-01-31 Thread Eric Blake
guestfs-release-notes-1.50.pod#pod-errors I presume there's a way to mark the .pod file, or at least tweak the Makefile that renders the page from .pod, to state that the input is intentionally UTF-8, in order to clear out that error. -- Eric Blake, Principal Software Engineer Red Hat, Inc.

Re: [Libguestfs] [PATCH nbdkit] curl: Enable multi-conn for read-only connections

2023-02-02 Thread Eric Blake
can_multi_conn: true > $ ./nbdkit curl file:/var/tmp/jammy-server-cloudimg-amd64.raw --run ' > nbdinfo $uri ' | grep can_multi_conn > can_multi_conn: false > > This improves performance. Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red

Re: [Libguestfs] Two multi-conn questions

2023-02-02 Thread Eric Blake
ds, it seems like qemu-img convert > above could go a lot faster if we allowed multi-conn. > > Is there any work on adding multi-conn support to qemu's NBD client? Not that I'm aware of at the moment, but we have proof that it may prove fruitful to have someone spend time on. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs

Re: [Libguestfs] Two multi-conn questions

2023-02-02 Thread Eric Blake
> real3m53.923s > user0m13.751s > sys 0m15.346s > > Now this is not something I'm personally concerned about (since I've > long been arguing we should deprecate the qemu curl driver and use > nbdkit), but

Re: [Libguestfs] [PATCH libnbd] generator: Pass LISTEN_FDNAMES=nbd with systemd socket activation

2023-02-06 Thread Eric Blake
On Sun, Feb 05, 2023 at 09:10:29AM +0100, Laszlo Ersek wrote: > On 2/1/23 17:17, Laszlo Ersek wrote: > > On 1/31/23 18:19, Eric Blake wrote: > > >> The alternative to relying on execvp() to scan PATH is to pre-scan > >> PATH ourselves before fork(). I wish t

Re: [Libguestfs] [libnbd PATCH v3 01/29] use space consistently in function and function-like macro invocations

2023-02-15 Thread Eric Blake
w; copy/file-ops.c:data = (char *) data + r; I think I'm a fan of having the space after the cast operator, and as long as we're doing tree-wide cleanups, this would be a good time to inject such a patch if we wanted. Also, it might be cool to have a code formatting tool automatically

Re: [Libguestfs] [libnbd PATCH v3 02/29] generator/C.ml: use space consistently in func. and func.-like macro calls

2023-02-15 Thread Eric Blake
n in this patch. And while I mentioned in 01/29 about the possibility of a C code formatter as a CI step for *.[ch], it would be even harder to insist that generated code matches a given style (it's always a nice goal for generated files to be human-readable where possible, but I'd mu

Re: [Libguestfs] [libnbd PATCH v3 03/29] socket activation: rename sa_(tmpdir|sockpath) to sact_(tmpdir|sockpath)

2023-02-15 Thread Eric Blake
-off-by: Laszlo Ersek > --- > lib/internal.h | 4 ++-- > generator/states-connect-socket-activation.c | 16 > lib/handle.c | 12 ++-- > 3 files changed, 16 insertions(+), 16 deletions(-) Reviewed-by: Eric Blake

Re: [Libguestfs] [libnbd PATCH v3 04/29] ocaml: rename "sa_u" to "saddr_u"

2023-02-15 Thread Eric Blake
by: Laszlo Ersek > --- > ocaml/helpers.c | 8 > 1 file changed, 4 insertions(+), 4 deletions(-) Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org __

Re: [Libguestfs] [libnbd PATCH v3 05/29] vector: (mostly) factor out DEFINE_VECTOR_EMPTY

2023-02-15 Thread Eric Blake
ector *env) > > err: >set_error (errno, "malloc"); > - string_vector_iter (env, (void *) free); > - free (env->ptr); > + string_vector_empty (env); >return -1; At any rate, I like how the callers look improved, even if we aren't quite settled on how the vector.h changes should end up. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org ___ Libguestfs mailing list Libguestfs@redhat.com https://listman.redhat.com/mailman/listinfo/libguestfs

Re: [Libguestfs] [libnbd PATCH v3 06/29] lib/utils: introduce xwrite() as a more robust write()

2023-02-15 Thread Eric Blake
al case, but it's good enough > for writing out diagnostics before giving up.) > > Signed-off-by: Laszlo Ersek > --- > lib/utils.c | 39 ++-- > 1 file changed, 35 insertions(+), 4 deletions(-) Reviewed-by: Eric Blake -- Eric Blake, Principal Software En

Re: [Libguestfs] [libnbd PATCH v3 07/29] lib/utils: add async-signal-safe assert()

2023-02-15 Thread Eric Blake
t(). But POSIX says that abort() shall attempt to override any SIGABRT handler in place, so I wonder if glibc's implementation is a bit too defensive. At any rate, your implementation looks reasonable, and more to the point, satisfies your desire of being async-signal-safe and thus appropriat

Re: [Libguestfs] [libnbd PATCH v3 08/29] lib/utils: add unit test for async-signal-safe assert()

2023-02-15 Thread Eric Blake
s would need to forward that on. > +signal_name=$(kill -l $exit_status) > +test "x$signal_name" = xABRT || test "x$signal_name" = xSIGABRT > + > +ptrn="^test-fork-safe-assert\\.c:[0-9]+: main: Assertion \`FALSE' > failed\\.\$&quo

Re: [Libguestfs] [libnbd PATCH v3 09/29] lib/utils: introduce async-signal-safe execvpe()

2023-02-15 Thread Eric Blake
cvpe_init(). Makes sense. A shame that it requires 3 functions, and that we have to duplicate so much work that libc would normally do for us if we don't care about async-safety, but I think the result is worth it. > > The main idea with the above distribution of responsibilities is tha

Re: [Libguestfs] [libnbd PATCH v3 10/29] lib/utils: add unit tests for async-signal-safe execvpe()

2023-02-15 Thread Eric Blake
fatal otherwise, we do get that error number. > +run empty:fifo:nxregf:symlink f > +execve_fail empty/f,fifo/f,nxregf/f,symlink/f ELOOP > + > +# Put a single prefix in PATH, such that it lead to a successful execution. > This leads > +# exercises two things at the same time: (a) t

Re: [Libguestfs] [libnbd PATCH v3 10/29] lib/utils: add unit tests for async-signal-safe execvpe()

2023-02-15 Thread Eric Blake
On Wed, Feb 15, 2023 at 05:03:48PM -0600, Eric Blake wrote: > > + > > +# Create subdirectories for triggering non-fatal internal error conditions > > of > > +# execvpe(). (Almost) every subdirectory will contain one entry, called > > "f". > > +# >

Re: [Libguestfs] [libnbd PATCH v3 11/29] socket activation: fix error message upon asprintf() failure

2023-02-15 Thread Eric Blake
t;malloc" would work too (we have several places where we aren't precise on WHAT allocation failed, but where it's obvious from the accompanying ENOMEM strerror() message that it was an allocation failure). But when it's easy to be precise, go for it. Reviewed-by: Eric Bl

Re: [Libguestfs] [PATCH 1/2] python: Avoid crash if callback parameters cannot be built

2023-02-17 Thread Eric Blake
with "N", not "O". That would be an alternative to decreasing the refcount of py_array on success, but not eliminate the need to decrease the refcount on Py_BuildValue failure. > > - when "args" is killed (decref'd), it takes

Re: [Libguestfs] [PATCH] python: Avoid leaking py_array along error paths

2023-02-17 Thread Eric Blake
"...O", py_array); Py_DECREF (py_array); if (args == NULL) { PyErr_PrintEx (0); goto out; } ... out: Py_DECREF (args); -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org

Re: [Libguestfs] [libnbd PATCH v3 01/29] use space consistently in function and function-like macro invocations

2023-02-17 Thread Eric Blake
On Fri, Feb 17, 2023 at 11:49:58AM +, Richard W.M. Jones wrote: > On Wed, Feb 15, 2023 at 01:48:39PM -0600, Eric Blake wrote: > > In my experience with GNU code (which this is not), the style I've > > seen there is to omit () whenever possible, as in: > > > >

Re: [Libguestfs] [libnbd PATCH v3 12/29] socket activation: clean up responsibilities of prep.sock.act.env.()

2023-02-17 Thread Eric Blake
>if (prepare_socket_activation_environment (&env) == -1) { > > SET_NEXT_STATE (%.DEAD); > > +set_error (errno, "prepare_socket_activation_environment"); > > Why move this out of the function? Moving it here lets us give a more specific message

Re: [Libguestfs] [libnbd PATCH v3 01/29] use space consistently in function and function-like macro invocations

2023-02-20 Thread Eric Blake
matically be enforced need not carry baggage of a large config file. But yeah, definitely not a priority for today. > > > Overall, I don't have any strong reasons to insist on shorter #ifdef > > spellings, and the rest of your changes, while mechanical, do make the > > codebas

Re: [Libguestfs] [libnbd PATCH v3 05/29] vector: (mostly) factor out DEFINE_VECTOR_EMPTY

2023-02-20 Thread Eric Blake
On Mon, Feb 20, 2023 at 06:03:13PM +0100, Laszlo Ersek wrote: > On 2/15/23 21:27, Eric Blake wrote: > > On Wed, Feb 15, 2023 at 03:11:34PM +0100, Laszlo Ersek wrote: > >> The "name##_iter" function is used 11 times in libnbd; in all those cases, > >> "name

Re: [Libguestfs] [libnbd PATCH v3 07/29] lib/utils: add async-signal-safe assert()

2023-02-20 Thread Eric Blake
On Mon, Feb 20, 2023 at 07:21:05PM +0100, Laszlo Ersek wrote: > On 2/15/23 21:57, Eric Blake wrote: > > On Wed, Feb 15, 2023 at 03:11:36PM +0100, Laszlo Ersek wrote: > >> Add an assert() variant that we may call between fork() and exec*(). > >> >

Re: [Libguestfs] [PATCH 1/2] python: Avoid crash if callback parameters cannot be built

2023-02-20 Thread Eric Blake
On Mon, Feb 20, 2023 at 09:08:08PM +0200, Nir Soffer wrote: > On Mon, Feb 20, 2023 at 10:45 AM Laszlo Ersek wrote: > > > > On 2/17/23 17:52, Eric Blake wrote: > > > On Thu, Feb 16, 2023 at 03:09:02PM +0100, Laszlo Ersek wrote: > > > > >> - Py_BuildValue w

Re: [Libguestfs] [libnbd PATCH v3 09/29] lib/utils: introduce async-signal-safe execvpe()

2023-02-21 Thread Eric Blake
On Tue, Feb 21, 2023 at 12:55:49PM +0100, Laszlo Ersek wrote: > On 2/15/23 23:28, Eric Blake wrote: > > On Wed, Feb 15, 2023 at 03:11:38PM +0100, Laszlo Ersek wrote: > >> execvp() [01] is powerful: > > > > You KNOW this is going to be a juicy commit message when t

Re: [Libguestfs] [libnbd PATCH v3 10/29] lib/utils: add unit tests for async-signal-safe execvpe()

2023-02-21 Thread Eric Blake
On Tue, Feb 21, 2023 at 02:06:33PM +0100, Laszlo Ersek wrote: > On 2/16/23 00:33, Eric Blake wrote: > > On Wed, Feb 15, 2023 at 05:03:48PM -0600, Eric Blake wrote: > >>> + > >>> +# Create subdirectories for triggering non-fatal internal error > >>> con

Re: [Libguestfs] [libnbd PATCH v3 10/29] lib/utils: add unit tests for async-signal-safe execvpe()

2023-02-21 Thread Eric Blake
On Tue, Feb 21, 2023 at 01:53:25PM +0100, Laszlo Ersek wrote: > On 2/16/23 00:03, Eric Blake wrote: > > On Wed, Feb 15, 2023 at 03:11:39PM +0100, Laszlo Ersek wrote: > >> Don't try to test async-signal-safety, but strive to exercise as many as > >> possible paths t

Re: [Libguestfs] [libnbd PATCH v3 12/29] socket activation: clean up responsibilities of prep.sock.act.env.()

2023-02-21 Thread Eric Blake
ctor', and so on, per each use of DEFINE_VECTOR_TYPE() - but while our one macro can easily declare other type-specific functions, you can't really use the preprocessor to define further type-specific macros. So having to tell the user to supply their own type name when reusing the type-

Re: [Libguestfs] [libnbd PATCH v3 09/29] lib/utils: introduce async-signal-safe execvpe()

2023-02-21 Thread Eric Blake
On Tue, Feb 21, 2023 at 07:07:38PM +0100, Laszlo Ersek wrote: > On 2/21/23 18:06, Eric Blake wrote: > > On Tue, Feb 21, 2023 at 12:55:49PM +0100, Laszlo Ersek wrote: > >> On 2/15/23 23:28, Eric Blake wrote: > >>> On Wed, Feb 15, 2023 at 03:11:38PM +0100, Laszlo Ersek

Re: [Libguestfs] [PATCH 1/2] python: Avoid crash if callback parameters cannot be built

2023-02-21 Thread Eric Blake
From: Eric Blake To: Laszlo Ersek , d...@python.org Cc: Nir Soffer , arye...@google.com, libguestfs@redhat.com, shtark...@google.com Bcc: Subject: Re: [Libguestfs] [PATCH 1/2] python: Avoid crash if callback parameters cannot be built Reply-To: In-Reply-To: [adding d...@python.org, in

Re: [Libguestfs] [libnbd PATCH v3 05/29] vector: (mostly) factor out DEFINE_VECTOR_EMPTY

2023-02-22 Thread Eric Blake
VECTOR_EMPTY_METHOD() instead of DEFINE_VECTOR_EMPTY() works for > > > me. > > > > OK, ADD_VECTOR_EMPTY_METHOD() can work with the above. > > This sounds fine for now, and since these are implementation details > we can always revisit them in future. -- Eric B

Re: [Libguestfs] [libnbd PATCH v3 09/29] lib/utils: introduce async-signal-safe execvpe()

2023-02-22 Thread Eric Blake
On Wed, Feb 22, 2023 at 01:47:02AM +0100, Laszlo Ersek wrote: > On 2/21/23 20:32, Eric Blake wrote: > > On Tue, Feb 21, 2023 at 07:07:38PM +0100, Laszlo Ersek wrote: > > >> Well, given Daniel's comments meanwhile, it seems like the original > >> execvp() i

[Libguestfs] [nbdkit PATCH] server: Don't assert on send if client hangs up early

2023-02-23 Thread Eric Blake
libnbd's copy/copy-nbd-error.sh was triggering an assertion failure in nbdkit: $ nbdcopy -- [ nbdkit --exit-with-parent -v --filter=error pattern 5M error-pread-rate=0.5 ] null: ... nbdkit: pattern.2: debug: error-inject: pread count=262144 offset=4718592 nbdkit: pattern.2: debug: pattern: pread

Re: [Libguestfs] [libnbd PATCH v4 3/5] remove semicolon after DEFINE_VECTOR_TYPE() macro invocations

2023-02-24 Thread Eric Blake
(exports, struct export); static exports export_list = empty_vector; static int diff --git c/info/map.c i/info/map.c index a5aad955..4924866a 100644 --- c/info/map.c +++ i/info/map.c @@ -36,7 +36,7 @@ #include "nbdinfo.h" -DEFINE_VECTOR_TYPE (uint32_vector, uint32_t) +DEFINE_VECTO

Re: [Libguestfs] [libnbd PATCH v4 0/5] vector: introduce DEFINE_POINTER_VECTOR_TYPE()

2023-02-24 Thread Eric Blake
or: introduce DEFINE_POINTER_VECTOR_TYPE() > convert string_vector_(iter(free) + reset()) to string_vector_empty() And turned out very nicely in my opinion. See my comments on 3. For 1,2,4,5: Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engine

Re: [Libguestfs] [libnbd PATCH v4 3/5] remove semicolon after DEFINE_VECTOR_TYPE() macro invocations

2023-02-24 Thread Eric Blake
On Fri, Feb 24, 2023 at 09:04:29AM -0600, Eric Blake wrote: > On Fri, Feb 24, 2023 at 05:39:35AM +0100, Laszlo Ersek wrote: > > A semicolon after a DEFINE_VECTOR_TYPE(...) macro invocation is not > > needed, nor does our documentation in "common/utils/vector.h" prescribe

Re: [Libguestfs] [nbdkit PATCH] server: Don't assert on send if client hangs up early

2023-02-24 Thread Eric Blake
On Fri, Feb 24, 2023 at 06:41:49AM +0100, Laszlo Ersek wrote: > On 2/23/23 21:40, Eric Blake wrote: > > libnbd's copy/copy-nbd-error.sh was triggering an assertion failure in > > nbdkit: > > > > $ nbdcopy -- [ nbdkit --exit-with-parent -v --filter=error pattern 5M

Re: [Libguestfs] [libnbd PATCH v3 07/29] lib/utils: add async-signal-safe assert()

2023-02-24 Thread Eric Blake
On Fri, Feb 24, 2023 at 01:57:37PM +0100, Laszlo Ersek wrote: > On 2/21/23 07:33, Laszlo Ersek wrote: > > On 2/20/23 19:21, Laszlo Ersek wrote: > >> On 2/15/23 21:57, Eric Blake wrote: > >>> On Wed, Feb 15, 2023 at 03:11:36PM +0100, Laszlo Ersek wrote: > > >

[Libguestfs] no way to force-close NBD handle in nbdsh

2023-02-24 Thread Eric Blake
the end, I did get a working test written (by just quit()ing python instead of trying to keep it alive). See my next email for the nbdkit patch that spawned my request here. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | l

[Libguestfs] [nbdkit PATCH v2] server: Don't assert on send if client hangs up early

2023-02-24 Thread Eric Blake
libnbd's copy/copy-nbd-error.sh was triggering an assertion failure in nbdkit (this is the command that triggered the failure, at least with nbdcopy 1.14.2; although future nbdcopy may be fixed independently to more gracefully exit): $ nbdcopy -- [ nbdkit --exit-with-parent -v --filter=error patte

Re: [Libguestfs] [nbdkit PATCH v2] server: Don't assert on send if client hangs up early

2023-02-27 Thread Eric Blake
On Sun, Feb 26, 2023 at 10:16:08AM +0100, Laszlo Ersek wrote: > On 2/24/23 23:59, Eric Blake wrote: > > @@ -752,14 +742,15 @@ protocol_recv_request_send_reply (void) > >(cmd == NBD_CMD_READ || cmd == NBD_CMD_BLOCK_STATUS)) { > > if (!error) { > >

Re: [Libguestfs] [libnbd PATCH v5 3/5] force semicolon after DEFINE_VECTOR_TYPE() macro invocations

2023-02-27 Thread Eric Blake
olons, but that doesn't play nicely with Emacs's C I'm not sure if the possessive is spelled "Emacs'". Either way, Reviewed-by: Eric Blake > language parser. Thus, force the semicolon instead. > > Signed-off-by: Laszlo Ersek > --- > > Notes: &g

Re: [Libguestfs] [libnbd PATCH v5 4/5] vector: introduce DEFINE_POINTER_VECTOR_TYPE()

2023-02-27 Thread Eric Blake
ring_vector_empty() in a subsequent patch. > > Signed-off-by: Laszlo Ersek > --- > > Notes: > v5: > > - resolve conflict from update to previous patch > > - force semicolon after ADD_VECTOR_EMPTY_METHOD() and > DEFINE_POINTER_VECTOR_TYPE() too >

Re: [Libguestfs] Checksums and other verification

2023-02-27 Thread Eric Blake
ally choose a hash that can be computed out-of-order, such as a Merkle Tree. But we'd need a standard setup for all parties to agree on how the hash is to be computed and checked, if it is going to be anything more than just a linear hash of the entire guest-visible conte

[Libguestfs] [nbdkit PATCH 0/5] ci: Get to green status on FreeBSD and MacOS

2023-03-01 Thread Eric Blake
but depends on an as-yet uncommitted patch in libvirt-ci: https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/360 Eric Blake (5): ci: Expose more env vars needed by build.sh ci: Another attempt at MacOS rust: Skip CI builds on MacOS golang: Skip CI builds on MacOS and newer FreeBSD perl:

[Libguestfs] [nbdkit PATCH 3/5] rust: Skip CI builds on MacOS

2023-03-01 Thread Eric Blake
rust is causing odd linker errors on MacOS 12: Compiling nbdkit v0.2.0 (/private/var/folders/76/zy5ktkns50v6gt5g8r0sf6scgn/T/cirrus-ci-build/plugins/rust) error: linking with `cc` failed: exit status: 1 | ... = note: Undefined symbols for architecture arm64: "_nbdkit_error", re

[Libguestfs] [nbdkit PATCH 2/5] ci: Another attempt at MacOS

2023-03-01 Thread Eric Blake
Import some lcitool improvements to avoid packages that are not available through homebrew on MacOS, as well as some other tweaks: - Bare libtorrent does not exist; libtorrent-rasterbar is available but fails to compile due to broken pkg-config --cflags - tcl is spelled tcl-tk - python3-boto3, sfd

[Libguestfs] [nbdkit PATCH 1/5] ci: Expose more env vars needed by build.sh

2023-03-01 Thread Eric Blake
In order for build.sh to see variables on Cirrus CI, we have to make sure they get passed through from the gitlab container. This will make it possible to skip language bindings that don't work on FreeBSD or MacOS. --- .gitlab-ci.yml | 3 +++ ci/cirrus/build.yml | 3 +++ 2 files changed, 6 i

[Libguestfs] [nbdkit PATCH 5/5] perl: Skip CI builds on newer FreeBSD

2023-03-01 Thread Eric Blake
perl is causing odd compiler errors, such as on FreeBSD 13: In file included from /usr/local/lib/perl5/5.32/mach/CORE/sbox32_hash.h:4: /usr/local/lib/perl5/5.32/mach/CORE/zaphod32_hash.h:164:5: error: '(' and '{' tokens introducing statement expression appear in different macro expansion context

[Libguestfs] [nbdkit PATCH 4/5] golang: Skip CI builds on MacOS and newer FreeBSD

2023-03-01 Thread Eric Blake
golang is causing odd linker errors on these platforms: FreeBSD 13: go build -o /tmp/cirrus-ci-build/plugins/golang/examples/disk/nbdkit-godisk-plugin.so -buildmode=c-shared # main /usr/local/go119/pkg/tool/freebsd_amd64/link: running cc failed: exit status 1 ld: error: /lib/libc.so.7: undefined

Re: [Libguestfs] [nbdkit PATCH 0/5] ci: Get to green status on FreeBSD and MacOS

2023-03-01 Thread Eric Blake
On Wed, Mar 01, 2023 at 06:43:11PM +0100, Laszlo Ersek wrote: > On 3/1/23 17:54, Eric Blake wrote: > > I took the easy route of crippling what I couldn't get working, on the > > grounds that partial coverage is better than none now that we have > > Cirrus CI chec

Re: [Libguestfs] [nbdkit PATCH 0/9] common: catch up with libnbd

2023-03-03 Thread Eric Blake
On Fri, Mar 03, 2023 at 07:00:16PM +, Richard W.M. Jones wrote: > > This series looks, great, thanks! > > ACK ACK from me as well; I didn't spot any problems in my read through. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtuali

[Libguestfs] [PATCH] docs: Prefer 'cookie' over 'handle'

2023-03-03 Thread Eric Blake
ke it obvious that a cookie is opaque data from the point of view of the server. Makes no difference to implementations (other than older code still using 'handle' may be slightly harder to tie back to the spec). Suggested-by: Richard W.M. Jones Signed-off-by: Eric Blake --- doc/

Re: [Libguestfs] [PATCH v2 1/6] spec: Recommend cap on NBD_REPLY_TYPE_BLOCK_STATUS length

2023-03-03 Thread Eric Blake
On Fri, Dec 16, 2022 at 10:32:01PM +0300, Vladimir Sementsov-Ogievskiy wrote: > On 11/15/22 01:46, Eric Blake wrote: > > The spec was silent on how many extents a server could reply with. > > However, both qemu and nbdkit (the two server implementations known to > >

Re: [Libguestfs] [PATCH v2 2/6] spec: Tweak description of maximum block size

2023-03-03 Thread Eric Blake
On Fri, Dec 16, 2022 at 11:22:49PM +0300, Vladimir Sementsov-Ogievskiy wrote: > On 11/15/22 01:46, Eric Blake wrote: > > Commit 9f30fedb improved the spec to allow non-payload requests that > > exceed any advertised maximum block size. Take this one step further > > by per

Re: [Libguestfs] [PATCH v2 2/6] spec: Tweak description of maximum block size

2023-03-03 Thread Eric Blake
st slow review time. > > On Mon, Nov 14, 2022 at 04:46:51PM -0600, Eric Blake wrote: > > Commit 9f30fedb improved the spec to allow non-payload requests that > > exceed any advertised maximum block size. Take this one step further > > by permitting the server to use

Re: [Libguestfs] [PATCH v2 3/6] spec: Add NBD_OPT_EXTENDED_HEADERS

2023-03-03 Thread Eric Blake
On Wed, Feb 22, 2023 at 11:49:18AM +0200, Wouter Verhelst wrote: > On Mon, Nov 14, 2022 at 04:46:52PM -0600, Eric Blake wrote: > [...] > > @@ -1370,9 +1475,10 @@ of the newstyle negotiation. > > Return a list of `NBD_REP_META_CONTEXT` replies, one per context, >

Re: [Libguestfs] [PATCH v2 5/6] spec: Introduce NBD_FLAG_BLOCK_STATUS_PAYLOAD

2023-03-03 Thread Eric Blake
On Wed, Feb 22, 2023 at 12:05:44PM +0200, Wouter Verhelst wrote: > On Mon, Nov 14, 2022 at 04:46:54PM -0600, Eric Blake wrote: > > Simple reply message > > > > @@ -1232,6 +1235,19 @@ The field has the following format: > >will be faster than a regular write

Re: [Libguestfs] [PATCH] docs: Prefer 'cookie' over 'handle'

2023-03-06 Thread Eric Blake
On Sun, Mar 05, 2023 at 10:53:38AM +0200, Wouter Verhelst wrote: > On Sat, Mar 04, 2023 at 10:03:46PM +0200, Nir Soffer wrote: > > On Sat, Mar 4, 2023 at 12:15 AM Eric Blake wrote: > > > Makes no difference to implementations (other than older code > > > still usi

Re: [Libguestfs] [PATCH libnbd] lib/errors.c: Fix assert fail in exit path in multi-threaded code

2023-03-08 Thread Eric Blake
zeof *last_error); > if (last_error) { >int err = pthread_setspecific (errors_key, last_error); > - if (err != 0) { > + /* Ignore EINVAL because that can happen in a race with other > + * threads when we are exiting. > + */ > + if (err

[Libguestfs] [libnbd PATCH v2] lib/errors.c: Fix assert fail in exit path in multi-threaded code

2023-03-08 Thread Eric Blake
ling exit(), to work around the assertion failure in nbdkit 1.32.5) - various fuse/* (not sure if we can address that; cleaning up FUSE is tricky) This reverts a small part of commit 831cbf7ee14c ("generator: Allow DEAD state actions to run"). Thanks: Richard W.M. Jones Signed-off-by

Re: [Libguestfs] [libnbd PATCH v2] lib/errors.c: Fix assert fail in exit path in multi-threaded code

2023-03-08 Thread Eric Blake
On Wed, Mar 08, 2023 at 04:29:03PM -0600, Eric Blake wrote: > > There aren't any good ways to fix this. We have proven that the > assertions are too tight (it IS possible for one thread's first use of > libnbd API, which causes the allocation of a thread-local error >

Re: [Libguestfs] [libnbd PATCH v2] lib/errors.c: Fix assert fail in exit path in multi-threaded code

2023-03-08 Thread Eric Blake
ly why libvirt uses -Z nodelete - the moment a thread can have thread-local data with a destructor pointing to libnbd code, but where libnbd can be removed from memory without calling pthread_key_delete(), is the moment you get a SEGV trying to call the thread destructor. A memory leak is

Re: [Libguestfs] [libnbd PATCH v4 1/3] lib/utils: introduce xwritel() as a more robust and convenient write()

2023-03-15 Thread Eric Blake
in nbd_internal_fork_safe_perror() where > at least one of the first two write() syscalls fails, and overwrites > "errno", before we get to formatting the error string from "errno". All nice benefits, even if we don't normally exercise the code. > > Than

Re: [Libguestfs] [libnbd PATCH v4 1/3] lib/utils: introduce xwritel() as a more robust and convenient write()

2023-03-15 Thread Eric Blake
On Wed, Mar 15, 2023 at 03:30:12PM +0100, Laszlo Ersek wrote: > On 3/15/23 15:01, Eric Blake wrote: > > > [...] > > Thanks for the thorough review; I'm glad all the fine points I sought to > put in the patch were received -- and well-received! :) > > One qu

Re: [Libguestfs] [libnbd PATCH v4 2/3] lib/utils: add async-signal-safe assert()

2023-03-15 Thread Eric Blake
LENO, file, ":", line_out, ": ", func, ": Assertion `", > + assertion, "' failed.\n", (char *)NULL); xwritel() makes this so much shorter ;) Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266

Re: [Libguestfs] [libnbd PATCH v4 3/3] lib/utils: add unit test for async-signal-safe assert()

2023-03-15 Thread Eric Blake
xported. On the other hand, prctl() is definitely Linux-specific, so I think you are quite safe in assuming that exists if and only if prctl() is a linkable entry point. If it does turn out to break someone, we can fix it in a followup patch, so no change needed in your usage at this time. >

Re: [Libguestfs] [libnbd PATCH v4 1/3] lib/utils: introduce xwritel() as a more robust and convenient write()

2023-03-17 Thread Eric Blake
nciple -- both > "libnbd.h" and "nbd-protocol.h" look public, at least to an extent, > while the stuff in (3.c) is totally internal. > > So, I can equate wrapper macros to public headers, for now, and I won't > introduce a new macro just for this one applica

Re: [Libguestfs] [libnbd PATCH v4 3/3] lib/utils: add unit test for async-signal-safe assert()

2023-03-17 Thread Eric Blake
On Thu, Mar 16, 2023 at 10:50:06AM +0100, Laszlo Ersek wrote: > On 3/15/23 18:25, Eric Blake wrote: > > On Wed, Mar 15, 2023 at 12:01:57PM +0100, Laszlo Ersek wrote: > >> Don't try to test async-signal-safety, only that > >> NBD_INTERNAL_FORK_SAFE_AS

Re: [Libguestfs] [libnbd PATCH v4 0/2] lib/utils: introduce async-signal-safe execvpe()

2023-03-20 Thread Eric Blake
irectory to it. > > +tmpd=$(mktemp -d) > > -+trap 'rm -r -- "$tmpd"' EXIT > > ++cleanup_fn rm -r -- "$tmpd" "$tmpd" probably starts with /, such that the -- is not strictly necessary; but keep it (good practice, and saves u

Re: [Libguestfs] [libnbd PATCH v4 0/2] lib/utils: introduce async-signal-safe execvpe()

2023-03-21 Thread Eric Blake
On Tue, Mar 21, 2023 at 07:04:59AM +0100, Laszlo Ersek wrote: > On 3/20/23 20:41, Eric Blake wrote: > > On Sun, Mar 19, 2023 at 10:41:37AM +0100, Laszlo Ersek wrote: > >> This is version 4 of the following sub-series: > >> > >> [libnbd PATCH v3 09/29] lib/utils:

Re: [Libguestfs] [libnbd PATCH v4 0/2] lib/utils: introduce async-signal-safe execvpe()

2023-03-21 Thread Eric Blake
for realpath deficiency on some platforms I didn't even pay attention to this one before you pointed it out; but it is indeed a bug in busybox now that POSIX is moving towards standardizing realpath, so I've filed it: https://bugs.busybox.net/show_bug.cgi?id=15466 > 2 65631e5dfff

Re: [Libguestfs] [libnbd PATCH v4 0/2] lib/utils: introduce async-signal-safe execvpe()

2023-03-22 Thread Eric Blake
On Wed, Mar 22, 2023 at 10:13:10AM +, Daniel P. Berrangé wrote: > On Tue, Mar 21, 2023 at 09:05:12AM -0500, Eric Blake wrote: > > > > $ podman build -f ci/containers/alpine-edge.Dockerfile -t libnbd-alpine-edge > > You usually shouldn't need this step when rep

Re: [Libguestfs] [libnbd PATCH v4 0/2] lib/utils: introduce async-signal-safe execvpe()

2023-03-22 Thread Eric Blake
On Wed, Mar 22, 2023 at 03:45:17PM +0100, Laszlo Ersek wrote: > On 3/21/23 18:28, Eric Blake wrote: > > > it is indeed a bug in busybox now that POSIX is moving towards > > standardizing realpath, so I've filed it: > > https://bugs.busybox.net/show_bug.cgi?id=15466 &g

Re: [Libguestfs] [libnbd PATCH v4 0/2] lib/utils: introduce async-signal-safe execvpe()

2023-03-22 Thread Eric Blake
ack when I have some URLs to list archives of my pending questions. > > This can be demonstrated with: > > $ PATH=.:$PATH strace -etrace=execve test-execvp > > execve("./test-execvp", ["test-execvp"], 0x7ffc0d1e5248 /* 85

Re: [Libguestfs] [libnbd PATCH v4 0/2] lib/utils: introduce async-signal-safe execvpe()

2023-03-22 Thread Eric Blake
researching for that, I found that FreeBSD had the same bug up until 2020: https://cgit.freebsd.org/src/commit/?id=301cb491ea On the other hand, the fact that FreeBSD changed and didn't suffer an immediate backlash of breaking programs means that glibc might consider such a change, despite the l

Re: [Libguestfs] [libnbd PATCH v4 0/2] lib/utils: introduce async-signal-safe execvpe()

2023-03-22 Thread Eric Blake
On Wed, Mar 22, 2023 at 03:07:01PM -0500, Eric Blake wrote: > On Wed, Mar 22, 2023 at 04:53:42PM +0100, Laszlo Ersek wrote: > > The solution is that glibc *too* has a bug, and that bug hides the busybox > > bug. Namely, in glibc, going back to historical commit

Re: [Libguestfs] [libnbd PATCH v3 03/19] socket activation: avoid manipulating the sign bit

2023-03-23 Thread Eric Blake
t warns about the short construct (at compile- or run-time), that would be a definitive reason to do this. But given that future standards will require the short form to work identically to the long form, and I'm unaware of a compiler that actually warns on the short form, I'm 50-50 on whe

Re: [Libguestfs] [libnbd PATCH v3 05/19] socket activation: centralize resource release

2023-03-23 Thread Eric Blake
nditional, but the condition would change to 'if (tmpdir)' instead of depending on 'next'. > + > +free_tmpdir: > + if (next == %.DEAD) > +free (tmpdir); And with transfer semantics and attribute cleanup, this is another label we could elide. > + > +done:

Re: [Libguestfs] [libnbd PATCH v3 07/19] socket activation: replace execvp() call with fork-safe variant

2023-03-23 Thread Eric Blake
perror (h->argv.ptr[0]); > if (errno == ENOENT) I had to check that nbd_internal_fork_safe_perror() preserves errno - but fortunately it does (it's always a good idea for functions designed to be used on cleanup handling paths to behave that way). Reviewed-by: Eric Blake -- Eric Blake

Re: [Libguestfs] [libnbd PATCH v3 11/19] CONNECT_COMMAND.START: sanitize close() calls in the child process

2023-03-23 Thread Eric Blake
calls, which are superfluous. > > Signed-off-by: Laszlo Ersek > Reviewed-by: Richard W.M. Jones > --- > Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org _

  1   2   3   4   5   6   7   8   9   10   >