David,
Thanks for including my email directly. It helps a lot.
Below, I'm going to assume that only "dest" is calling
qemu_rdma_exchange_recv()
and only src is calling qemu_rdma_exchange_send(), since you didn't specify
who
is sending and who is receiving.
If that assumption is wrong, please respond again.
Comments inline.....
On Sat, Dec 12, 2015 at 1:48 AM, Dr. David Alan Gilbert <dgilb...@redhat.com
> wrote:
> Hi Michael,
> I think I've got an RDMA race condition, but I'm being a little
> cautious at the moment and wondered if you agree with the following
> diagnosis.
>
> It's showing up in a world of mine that's sending more control messages
> from the destination->source and I'm seeing the following.
>
> We normally expect:
>
> src dest
> ----------->control ready->
>
If src is sending, this is not correct. Dest should send the ready message
if it is receiving, not src, which breaks the above assumption. So, I'll
reverse the assumption previously and continue with your observation and
assume that src is receiving instead of dest, which should instead look
like:
src (receiving) dest (sending)
----------->control ready->
> Sees SEND_CONTROL signal to ack that it has been sent
>
I'll assume here that you meant that dest sees the ready message and is
then later sends something.
> <-----control message--
> Sees RECV_CONTROL message from dest
>
>
Similar assumption for the receiver (src).
> but what I'm seeing is:
> src dest
> ----------->control ready->
> <-----control message--
> Sees RECV_CONTROL message from dest
>
hmmmmm....
> Sees SEND_CONTROL signal to ack that it has been sent
>
>
There's not enough information here....... do you have a multi-threaded
send or receive or something?
Do the work request IDs match up?
- Michael
