On Mon, 7 Jan 2019 19:28:10 +0200 Marcel Apfelbaum <marcel.apfelb...@gmail.com> wrote:
> On 1/3/19 12:34 PM, Cornelia Huck wrote: > > On Sat, 22 Dec 2018 11:50:07 +0200 > > Marcel Apfelbaum <marcel.apfelb...@gmail.com> wrote: > > > >> From: Yuval Shaia <yuval.sh...@oracle.com> > >> > >> RDMA MAD kernel module (ibcm) disallow more than one MAD-agent for a > >> given MAD class. > >> This does not go hand-by-hand with qemu pvrdma device's requirements > >> where each VM is MAD agent. > >> Fix it by adding implementation of RDMA MAD multiplexer service which on > >> one hand register as a sole MAD agent with the kernel module and on the > >> other hand gives service to more than one VM. > >> > >> Design Overview: > >> Reviewed-by: Shamir Rabinovitch <shamir.rabinovi...@oracle.com> > >> ---------------- > >> A server process is registered to UMAD framework (for this to work the > >> rdma_cm kernel module needs to be unloaded) and creates a unix socket to > >> listen to incoming request from clients. > >> A client process (such as QEMU) connects to this unix socket and > >> registers with its own GID. > >> > >> TX: > >> ---- > >> When client needs to send rdma_cm MAD message it construct it the same > >> way as without this multiplexer, i.e. creates a umad packet but this > >> time it writes its content to the socket instead of calling umad_send(). > >> The server, upon receiving such a message fetch local_comm_id from it so > >> a context for this session can be maintain and relay the message to UMAD > >> layer by calling umad_send(). > >> > >> RX: > >> ---- > >> The server creates a worker thread to process incoming rdma_cm MAD > >> messages. When an incoming message arrived (umad_recv()) the server, > >> depending on the message type (attr_id) looks for target client by > >> either searching in gid->fd table or in local_comm_id->fd table. With > >> the extracted fd the server relays to incoming message to the client. > >> > >> Signed-off-by: Yuval Shaia <yuval.sh...@oracle.com> > >> Reviewed-by: Shamir Rabinovitch <shamir.rabinovi...@oracle.com> > >> Signed-off-by: Marcel Apfelbaum <marcel.apfelb...@gmail.com> > >> > >> Signed-off-by: Marcel Apfelbaum <marcel.apfelb...@gmail.com> > >> --- > >> MAINTAINERS | 1 + > >> Makefile | 3 + > >> Makefile.objs | 1 + > >> contrib/rdmacm-mux/Makefile.objs | 4 + > >> contrib/rdmacm-mux/main.c | 798 +++++++++++++++++++++++++++++++ > >> contrib/rdmacm-mux/rdmacm-mux.h | 61 +++ > >> 6 files changed, 868 insertions(+) > >> create mode 100644 contrib/rdmacm-mux/Makefile.objs > >> create mode 100644 contrib/rdmacm-mux/main.c > >> create mode 100644 contrib/rdmacm-mux/rdmacm-mux.h > > This commit breaks building with clang on Fedora 28 for me; I get a > > bunch of > > > > Hi Cornelia, > Thanks for spotting the issue. > > > clang-6.0: error: -libumad: 'linker' input unused > > [-Werror,-Wunused-command-line-argument] > > Is really strange, the rdma-mux is part of the contrib directory > and is not even compiled by default. > > > Explicitly passing --disable-pvrdma to configure disables the offending > > code. > > > > Let me know if you need more information. > > Can you please send the exact steps you are using to configure and > compile QEMU, > are you compiling on a x86 machine? Yes, this is on my laptop (up-to-date F28). I can reproduce on current master, configured with ../configure --target-list="s390x-softmmu s390x-linux-user i386-softmmu x86_64-softmmu cris-softmmu arm-softmmu ppc64-softmmu" --enable-linux-aio --enable-virtfs --enable-trace-backends=simple --cc=clang --enable-rdma --enable-pvrdma It compiles if I pass --disable-pvrdma instead. It also compiles if I drop --cc=clang. I'm building in a build subdirectory; I have already tried purging it before rebuilding, without luck. clang --version says clang version 6.0.1 (tags/RELEASE_601/final) Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /usr/bin
