Hi Ralph,
On Fri, 23 Apr 2021 08:56:48 +0200
Ralph Schmieder <ralph.schmie...@gmail.com> wrote:
> Hey... new to this list. I was looking for a way to use Unix domain
> sockets as a network transport between local VMs.
>
> I'm part of a team where we run dozens if not hundreds of VMs on a
> single compute instance which are highly interconnected.
>
> In the current implementation, I use UDP sockets (e.g. something like
>
> -netdev
> id=bla,type=socket,udp=localhost:1234,localaddr=localhost:5678)
>
> which works great.
>
> The downside of this approach is that I need to keep track of all the
> UDP ports in use and that there's a potential for clashes. Clearly,
> having Unix domain sockets where I could store the sockets in the
> VM's directory would be much easier to manage.
>
> However, even though there is some AF_UNIX support in net/socket.c,
> it's
>
> - not configurable
> - it doesn't work
I hate to say this, but I've been working on something very similar
just in the past days, with the notable difference that I'm using
stream-oriented AF_UNIX sockets instead of datagram-oriented.
I have a similar use case, and after some experiments I picked a
stream-oriented socket over datagram-oriented because:
- overhead appears to be the same
- message boundaries make little sense here -- you already have a
32-bit vnet header with the message size defining the message
boundaries
- datagram-oriented AF_UNIX sockets are actually reliable and there's
no packet reordering on Linux, but this is not "formally" guaranteed
- it's helpful for me to know when a qemu instance disconnects for any
reason
> As a side note, I tried to pass in an already open FD, but that
> didn't work either.
This actually worked for me as a quick work-around, either with:
https://github.com/StevenVanAcker/udstools
or with a trivial C implementation of that, that does essentially:
fd = strtol(argv[1], NULL, 0);
if (fd < 3 || fd > INT_MAX || errno)
usage(argv[0]);
s = socket(AF_UNIX, SOCK_STREAM, 0);
if (s < 0) {
perror("socket");
exit(EXIT_FAILURE);
}
if (connect(s, (const struct sockaddr *)&addr, sizeof(addr)) < 0) {
perror("connect");
exit(EXIT_FAILURE);
}
if (dup2(s, (int)fd) < 0) {
perror("dup");
exit(EXIT_FAILURE);
}
close(s);
execvp(argv[2], argv + 2);
perror("execvp");
where argv[1] is the socket number you pass in the qemu command line
(-net socket,fd=X) and argv[2] is the path to qemu.
> So, I added some code which does work for me... e.g.
>
> - can specify the socket paths like -netdev
> id=bla,type=socket,unix=/tmp/in:/tmp/out
> - it does forward packets between two Qemu instances running
> back-to-back
>
> I'm wondering if this is of interest for the wider community and, if
> so, how to proceed.
>
> Thanks,
> -ralph
>
> Commit
> https://github.com/rschmied/qemu/commit/73f02114e718ec898c7cd8e855d0d5d5d7abe362
I think your patch could be a bit simpler, as you could mostly reuse
net_socket_udp_init() for your initialisation, and perhaps rename it to
net_socket_dgram_init().
Anyway, I wonder, would my approach work for you instead? I'm posting my
patch in a minute.
--
Stefano
