On 05.03.2013 04:21, Lawrence Stewart wrote:
On 03/05/13 03:35, Andre Oppermann wrote:
On 26.02.2013 14:38, Lawrence Stewart wrote:
Hi Andre,
Hi Lawrence, :-)
A colleague and I spent a very frustrating day tracing an accounting bug
in the multipath TCP patch we're working on at CAIA to a bug in
sbsndptr(). I haven't tested it with regular TCP yet, but I believe the
following patch fixes the bug (proposed commit log message is at the top
of the patch):
http://people.freebsd.org/~lstewart/patches/misctcp/sbsndptr_mnext_10.x.r247314.diff
The patch should have no tangible effect to operation other than to
ensure the function delivers on the promise to return the closest mbuf
in the chain for the given offset.
I agree that the description of sbsndptr() can be misleading as it refers
to the point in time when the pointer was updated last. Relative to now
the real offset may be at the beginning of the next mbuf.
Right, and we ran into the issue because we made an assumption based on
the use of the present tense in the comment:
"Return closest mbuf in chain for current offset."
I apologize for the incorrect and misleading description. :-)
As you note in the proposed commit message by the time the send pointer
is calculated we may have reached the end of the chain and must avoid
storing a NULL pointer. The mbuf copy routines simply skips over the
additional mbuf in the chain using the returned offset.
I wonder how this has caused trouble with your multipath patch. You'd
have to copy the sockbuf contents as well and unless you're using custom
sockbuf and mbuf chain functions this shouldn't be a problem. Using
custom functions on a socket buffer is a delicate approach. For a sockbuf
consumer being able to handle valid offsets into an mbuf chain is a core
feature and must-have part of the functionality.
No custom sockbuf or mbuf routines are in use. We've implemented a
mapping shim between subflows and the socket buffer. When a subflow asks
the multipath layer for some data to send, the multipath layer returns a
mapping onto the socket buffer, which will remain valid until such time
as the subflow has marked the mapped data as acknowledged.
>
Part of the map accounting is tracking the pointer of the first mbuf in
the sockbuf where the map's data begins. Our accounting assumed the mbuf
+ the offset returned by sbsndptr had data available, which is how we
triggered the problem. We could have accounted for the issue in our new
map accounting code, but that would add additional complexity to some
already complex code and the better solution is to make sbsndptr DTRT.
So effectively you run a separate sbsndptr for each subflow using the
real sbsndptr to track the head of the queue?
/me fears the day a mptcp import comes up. tcp-complexity^^3. :-o
I would appreciate a review and any thoughts.
I think you have found a valid (micro-)optimization. However you're
still making a dangerous assumption in that the next mbuf is indeed
the one you want. This may not be true in subtle ways when the chain
contains m_len=0 mbufs in it. I'm not aware of it actually happening
but it can't be ruled out either if custom sockbuf manipulation functions
are in use.
True, though I'm struggling to think why there would be m_len=0 mbufs
interspersed with m_len > 0 mbufs in a socket send buffer mbuf chain.
sbcompress() doesn't allow for m_len=0 mbufs. This holds true as long
as the sbappend functions are used. If not, we may get anything there.
As long as nobody is using custom sockbuf appends we're safe. Because
I first assumed from your description some custom sockbuf munging the
guarantee wouldn't haven been there anymore.
I'd recommend the following:
have you custom sockbuf function handle forward seeking like the other
m_copy() functions; and/or apply a patch along the (untested) example
below.
If you believe it is both correct and possible for m_len=0 mbufs to
exist in a socket buffer chain, then I agree that we should amend my
proposed patch to loop and skip over m_len=0 mbufs as you've suggested.
No. So far it is neither possible or correct.
However, I'm more inclined to suspect it is undesirable and potentially
buggy behaviour to end up with m_len=0 mbufs in a socket buffer chain on
which sbsndptr is being used, and would instead suggest a
"KASSERT(ret->m_len > 0, (...));" be added to the end of my proposed if
block.
Agreed.
--
Andre
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
