Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-09-27 Thread Richard W.M. Jones
On Tue, Sep 27, 2022 at 10:11:24PM +0800, Ming Lei wrote: > On Tue, Sep 27, 2022 at 10:42:45AM +0100, Richard W.M. Jones wrote: > > On Wed, Sep 14, 2022 at 09:47:49AM +0800, Ming Lei wrote: > > > FYI, I plan to stabilize libublksrv's API in 1.0 release, which > > > shouldn't take long. > > > > > >

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-09-27 Thread Ming Lei
On Tue, Sep 27, 2022 at 10:42:45AM +0100, Richard W.M. Jones wrote: > On Wed, Sep 14, 2022 at 09:47:49AM +0800, Ming Lei wrote: > > FYI, I plan to stabilize libublksrv's API in 1.0 release, which > > shouldn't take long. > > > > And recently I did some API changes, so suggest you to update with >

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-09-27 Thread Richard W.M. Jones
On Tue, Sep 27, 2022 at 10:42:45AM +0100, Richard W.M. Jones wrote: > On Wed, Sep 14, 2022 at 09:47:49AM +0800, Ming Lei wrote: > > FYI, I plan to stabilize libublksrv's API in 1.0 release, which > > shouldn't take long. > > > > And recently I did some API changes, so suggest you to update with >

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-09-27 Thread Richard W.M. Jones
On Wed, Sep 14, 2022 at 09:47:49AM +0800, Ming Lei wrote: > FYI, I plan to stabilize libublksrv's API in 1.0 release, which > shouldn't take long. > > And recently I did some API changes, so suggest you to update with > it in both nbdublk and libublksrv. Just got around to looking at the changes

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-09-13 Thread Ming Lei
On Fri, Sep 02, 2022 at 10:42:41AM +0100, Richard W.M. Jones wrote: > FYI, nbdublk is now available in Fedora Rawhide, along with the > dependencies liburing 2.2 and ubdsrv. > > https://koji.fedoraproject.org/koji/buildinfo?buildID=2056223 > > Thanks for your help. Hi Jones, FYI, I plan to stab

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-09-05 Thread Richard W.M. Jones
On Mon, Sep 05, 2022 at 08:06:24PM +0800, Ming Lei wrote: > On Fri, Sep 02, 2022 at 09:06:27AM +0100, Richard W.M. Jones wrote: > > On Thu, Sep 01, 2022 at 12:14:40PM +0100, Richard W.M. Jones wrote: > > > On Thu, Sep 01, 2022 at 04:01:39PM +0800, Ming Lei wrote: > > > > Maybe you should use one nb

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-09-05 Thread Ming Lei
On Fri, Sep 02, 2022 at 09:06:27AM +0100, Richard W.M. Jones wrote: > On Thu, Sep 01, 2022 at 12:14:40PM +0100, Richard W.M. Jones wrote: > > On Thu, Sep 01, 2022 at 04:01:39PM +0800, Ming Lei wrote: > > > Maybe you should use one nbd disk, which has the closest code path > > > with nbdublk. > > >

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-09-02 Thread Richard W.M. Jones
FYI, nbdublk is now available in Fedora Rawhide, along with the dependencies liburing 2.2 and ubdsrv. https://koji.fedoraproject.org/koji/buildinfo?buildID=2056223 Thanks for your help. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming a

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-09-02 Thread Richard W.M. Jones
On Thu, Sep 01, 2022 at 12:14:40PM +0100, Richard W.M. Jones wrote: > On Thu, Sep 01, 2022 at 04:01:39PM +0800, Ming Lei wrote: > > Maybe you should use one nbd disk, which has the closest code path > > with nbdublk. > > Good idea - I have started a heavy test using nbd.ko as the > backing. Let's

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-09-01 Thread Ming Lei
On Thu, Sep 01, 2022 at 12:14:40PM +0100, Richard W.M. Jones wrote: > On Thu, Sep 01, 2022 at 04:01:39PM +0800, Ming Lei wrote: > > Maybe you should use one nbd disk, which has the closest code path > > with nbdublk. > > Good idea - I have started a heavy test using nbd.ko as the > backing. Let's

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-09-01 Thread Richard W.M. Jones
On Thu, Sep 01, 2022 at 04:01:39PM +0800, Ming Lei wrote: > Maybe you should use one nbd disk, which has the closest code path > with nbdublk. Good idea - I have started a heavy test using nbd.ko as the backing. Let's see what happens after 12 hours or more. Rich. -- Richard Jones, Virtualizat

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-09-01 Thread Ming Lei
On Wed, Aug 31, 2022 at 12:06:37PM +0100, Richard W.M. Jones wrote: > On Wed, Aug 31, 2022 at 06:33:25PM +0800, Ming Lei wrote: > > On Wed, Aug 31, 2022 at 11:22:04AM +0100, Richard W.M. Jones wrote: > > > > > > After running those tests for an hour or two I got a nasty kernel > > > memory corrupt

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-09-01 Thread Richard W.M. Jones
On Thu, Sep 01, 2022 at 11:26:10AM +0800, Ming Lei wrote: > On Wed, Aug 31, 2022 at 04:06:16PM +0100, Richard W.M. Jones wrote: > > I have also added the changes to Fedora: > > > > https://koji.fedoraproject.org/koji/taskinfo?taskID=91468555 > > > > Please consider becoming a Fedora packager so y

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-09-01 Thread Richard W.M. Jones
On Wed, Aug 31, 2022 at 11:22:04AM +0100, Richard W.M. Jones wrote: > > After running those tests for an hour or two I got a nasty kernel > memory corruption issue in the test VM, see attached. Another one from running overnight. Rich. -- Richard Jones, Virtualization Group, Red Hat http://peo

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-31 Thread Ming Lei
On Wed, Aug 31, 2022 at 04:06:16PM +0100, Richard W.M. Jones wrote: > On Wed, Aug 31, 2022 at 10:49:42PM +0800, Ming Lei wrote: > > On Wed, Aug 31, 2022 at 5:41 PM Richard W.M. Jones > > wrote: > > > > > > On Wed, Aug 31, 2022 at 05:29:13PM +0800, Ming Lei wrote: > > > > I can reproduce it, pleas

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-31 Thread Richard W.M. Jones
On Wed, Aug 31, 2022 at 10:49:42PM +0800, Ming Lei wrote: > On Wed, Aug 31, 2022 at 5:41 PM Richard W.M. Jones wrote: > > > > On Wed, Aug 31, 2022 at 05:29:13PM +0800, Ming Lei wrote: > > > I can reproduce it, please try the top patch in aio branch, which fixed > > > hang in my reproducer with you

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-31 Thread Ming Lei
On Wed, Aug 31, 2022 at 5:41 PM Richard W.M. Jones wrote: > > On Wed, Aug 31, 2022 at 05:29:13PM +0800, Ming Lei wrote: > > I can reproduce it, please try the top patch in aio branch, which fixed > > hang in my reproducer with your test setting. > > > > https://github.com/ming1/ubdsrv/commits/aio

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-31 Thread Richard W.M. Jones
On Wed, Aug 31, 2022 at 06:33:25PM +0800, Ming Lei wrote: > On Wed, Aug 31, 2022 at 11:22:04AM +0100, Richard W.M. Jones wrote: > > > > After running those tests for an hour or two I got a nasty kernel > > memory corruption issue in the test VM, see attached. > > >From the dmesg log, the direct i

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-31 Thread Ming Lei
On Wed, Aug 31, 2022 at 11:22:04AM +0100, Richard W.M. Jones wrote: > > After running those tests for an hour or two I got a nasty kernel > memory corruption issue in the test VM, see attached. >From the dmesg log, the direct issue is from zram. The userspace ublksrv shouldn't trigger this kind

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-31 Thread Richard W.M. Jones
After running those tests for an hour or two I got a nasty kernel memory corruption issue in the test VM, see attached. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets y

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-31 Thread Richard W.M. Jones
On Wed, Aug 31, 2022 at 05:29:13PM +0800, Ming Lei wrote: > I can reproduce it, please try the top patch in aio branch, which fixed > hang in my reproducer with your test setting. > > https://github.com/ming1/ubdsrv/commits/aio (https://github.com/ming1/ubdsrv/commit/0a293b6eb7149dc5ee83e5d07d242

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-31 Thread Ming Lei
On Wed, Aug 31, 2022 at 10:13:08AM +0100, Richard W.M. Jones wrote: > On Wed, Aug 31, 2022 at 09:45:45AM +0800, Ming Lei wrote: > > On Tue, Aug 30, 2022 at 05:13:46PM +0100, Richard W.M. Jones wrote: > > > On Tue, Aug 30, 2022 at 11:29:26PM +0800, Ming Lei wrote: > > > > On Tue, Aug 30, 2022 at 03:

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-31 Thread Richard W.M. Jones
On Wed, Aug 31, 2022 at 09:45:45AM +0800, Ming Lei wrote: > On Tue, Aug 30, 2022 at 05:13:46PM +0100, Richard W.M. Jones wrote: > > On Tue, Aug 30, 2022 at 11:29:26PM +0800, Ming Lei wrote: > > > On Tue, Aug 30, 2022 at 03:38:50PM +0100, Richard W.M. Jones wrote: > > > > On Tue, Aug 30, 2022 at 03:

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-30 Thread Ming Lei
On Tue, Aug 30, 2022 at 05:13:46PM +0100, Richard W.M. Jones wrote: > On Tue, Aug 30, 2022 at 11:29:26PM +0800, Ming Lei wrote: > > On Tue, Aug 30, 2022 at 03:38:50PM +0100, Richard W.M. Jones wrote: > > > On Tue, Aug 30, 2022 at 03:12:23PM +0800, Ming Lei wrote: > > > > The patch sent in last emai

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-30 Thread Richard W.M. Jones
On Tue, Aug 30, 2022 at 11:29:26PM +0800, Ming Lei wrote: > On Tue, Aug 30, 2022 at 03:38:50PM +0100, Richard W.M. Jones wrote: > > On Tue, Aug 30, 2022 at 03:12:23PM +0800, Ming Lei wrote: > > > The patch sent in last email may cause io hang on MQ, and follows the > > > fixed > > > version: > >

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-30 Thread Ming Lei
On Tue, Aug 30, 2022 at 11:29:26PM +0800, Ming Lei wrote: > On Tue, Aug 30, 2022 at 03:38:50PM +0100, Richard W.M. Jones wrote: > > On Tue, Aug 30, 2022 at 03:12:23PM +0800, Ming Lei wrote: > > > The patch sent in last email may cause io hang on MQ, and follows the > > > fixed > > > version: > >

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-30 Thread Ming Lei
On Tue, Aug 30, 2022 at 03:38:50PM +0100, Richard W.M. Jones wrote: > On Tue, Aug 30, 2022 at 03:12:23PM +0800, Ming Lei wrote: > > The patch sent in last email may cause io hang on MQ, and follows the fixed > > version: > > I split this into two commits and cleaned them up and posted them here: >

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-30 Thread Richard W.M. Jones
On Tue, Aug 30, 2022 at 03:12:23PM +0800, Ming Lei wrote: > The patch sent in last email may cause io hang on MQ, and follows the fixed > version: I split this into two commits and cleaned them up and posted them here: https://gitlab.com/rwmjones/libnbd/-/commits/nbdublk/ Unfortunately this does

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-30 Thread Ming Lei
On Tue, Aug 30, 2022 at 11:51:00AM +0100, Richard W.M. Jones wrote: > On Tue, Aug 30, 2022 at 05:38:35PM +0800, Ming Lei wrote: > > nbd work thread is created by nbd target code just like before, but > > the thread is changed to the following way, basically bound with one > > aio_ctx: > > > whil

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-30 Thread Richard W.M. Jones
On Tue, Aug 30, 2022 at 05:38:35PM +0800, Ming Lei wrote: > nbd work thread is created by nbd target code just like before, but > the thread is changed to the following way, basically bound with one > aio_ctx: > while (!ublksrv_aio_ctx_dead(aio_ctx)) { > struct aio_list compl; > > a

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-30 Thread Ming Lei
On Tue, Aug 30, 2022 at 10:07:40AM +0100, Richard W.M. Jones wrote: > On Tue, Aug 30, 2022 at 04:30:40PM +0800, Ming Lei wrote: > > On Tue, Aug 30, 2022 at 09:04:07AM +0100, Richard W.M. Jones wrote: > > > On Tue, Aug 30, 2022 at 10:32:02AM +0800, Ming Lei wrote: > > > > Hi Jones, > > > > > > > >

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-30 Thread Richard W.M. Jones
On Tue, Aug 30, 2022 at 04:30:40PM +0800, Ming Lei wrote: > On Tue, Aug 30, 2022 at 09:04:07AM +0100, Richard W.M. Jones wrote: > > On Tue, Aug 30, 2022 at 10:32:02AM +0800, Ming Lei wrote: > > > Hi Jones, > > > > > > On Thu, Aug 25, 2022 at 01:10:55PM +0100, Richard W.M. Jones wrote: > > > > This

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-30 Thread Ming Lei
On Tue, Aug 30, 2022 at 09:04:07AM +0100, Richard W.M. Jones wrote: > On Tue, Aug 30, 2022 at 10:32:02AM +0800, Ming Lei wrote: > > Hi Jones, > > > > On Thu, Aug 25, 2022 at 01:10:55PM +0100, Richard W.M. Jones wrote: > > > This patch adds simple support for a ublk-based NBD client. > > > It is al

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-30 Thread Richard W.M. Jones
On Tue, Aug 30, 2022 at 10:32:02AM +0800, Ming Lei wrote: > Hi Jones, > > On Thu, Aug 25, 2022 at 01:10:55PM +0100, Richard W.M. Jones wrote: > > This patch adds simple support for a ublk-based NBD client. > > It is also available here: > > https://gitlab.com/rwmjones/libnbd/-/tree/nbdublk/ublk >

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-30 Thread Ming Lei
On Tue, Aug 30, 2022 at 10:32:02AM +0800, Ming Lei wrote: > Hi Jones, > > On Thu, Aug 25, 2022 at 01:10:55PM +0100, Richard W.M. Jones wrote: > > This patch adds simple support for a ublk-based NBD client. > > It is also available here: > > https://gitlab.com/rwmjones/libnbd/-/tree/nbdublk/ublk >

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-30 Thread Ming Lei
Hi Jones, On Thu, Aug 25, 2022 at 01:10:55PM +0100, Richard W.M. Jones wrote: > This patch adds simple support for a ublk-based NBD client. > It is also available here: > https://gitlab.com/rwmjones/libnbd/-/tree/nbdublk/ublk > > ublk is a way to write Linux block device drivers in userspace: Ju

Re: [Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-25 Thread Richard W.M. Jones
FYI here is the asynch version: https://gitlab.com/rwmjones/libnbd/-/commits/nbdublk/ I kept it as two commits so you can see the differences but obviously it would be folded into one commit upstream. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read m

[Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-25 Thread Richard W.M. Jones
--- .gitignore | 3 + Makefile.am | 3 +- README.md | 2 + bash-completion/Makefile.am | 8 +- bash-completion/nbdsh | 6 + configure.ac| 20 ++ copy/nbdcopy.pod| 1 + docs/libnbd.pod |

[Libguestfs] [PATCH libnbd] ublk: Add new nbdublk program

2022-08-25 Thread Richard W.M. Jones
This patch adds simple support for a ublk-based NBD client. It is also available here: https://gitlab.com/rwmjones/libnbd/-/tree/nbdublk/ublk ublk is a way to write Linux block device drivers in userspace: https://lwn.net/Articles/903855/ For simplicity of implementation and because I don't curre