Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-25 Thread Sowmini Varadhan
On (11/24/15 17:25), Florian Westphal wrote: > Its a well-written document, but I don't see how moving the burden of > locking a single logical tcp connection (to prevent threads from > reading a partial record) from userspace to kernel is an improvement. > > If you really have 100 threads and mus

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-25 Thread Hannes Frederic Sowa
Hi, On Tue, Nov 24, 2015, at 23:21, Alexei Starovoitov wrote: > On Tue, Nov 24, 2015 at 10:58:08PM +0100, Hannes Frederic Sowa wrote: > > > > > > > > > > interesting idea, but then remote host will be influencing local cpu > > > > > selection? > > > > > how remote can figure out the number of loc

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Hannes Frederic Sowa
Hi David, On Tue, Nov 24, 2015, at 23:25, David Miller wrote: > From: Florian Westphal > Date: Tue, 24 Nov 2015 23:22:42 +0100 > > > Yes, I get that point, but I maintain that KCM is a strange workaround > > for bad userspace design. > > I fundamentally disagree with you. > > And even if I did

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Florian Westphal
David Miller wrote: > From: Florian Westphal > Date: Tue, 24 Nov 2015 23:22:42 +0100 > > > Yes, I get that point, but I maintain that KCM is a strange workaround > > for bad userspace design. > > I fundamentally disagree with you. Fair enough. Still, I do not see how what KCM intends to do ca

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread David Miller
From: Florian Westphal Date: Tue, 24 Nov 2015 23:22:42 +0100 > Yes, I get that point, but I maintain that KCM is a strange workaround > for bad userspace design. I fundamentally disagree with you. And even if I didn't, I would be remiss to completely dismiss the difficulty in changing existing

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Florian Westphal
Tom Herbert wrote: > On Tue, Nov 24, 2015 at 12:55 PM, Florian Westphal wrote: > > Why anyone would invest such a huge amount of work in making this > > kernel-based framing for single-stream tcp record (de)mux rather than > > improving the userspace protocol to use UDP or SCTP or at least > > on

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Tom Herbert
On Tue, Nov 24, 2015 at 12:55 PM, Florian Westphal wrote: > Tom Herbert wrote: >> Message size limits can be enforced in BPF or we could add a limit >> enforced by KCM. For instance, the message size limit in http/2 is >> 16M. If it's needed, it wouldn't be much trouble to add a streaming >> inte

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Florian Westphal
Tom Herbert wrote: > Message size limits can be enforced in BPF or we could add a limit > enforced by KCM. For instance, the message size limit in http/2 is > 16M. If it's needed, it wouldn't be much trouble to add a streaming > interface for large messages. That still won't change the fact that

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Alexei Starovoitov
On Tue, Nov 24, 2015 at 08:16:25PM +0100, Hannes Frederic Sowa wrote: > Hello, > > On Tue, Nov 24, 2015, at 19:59, Alexei Starovoitov wrote: > > On Tue, Nov 24, 2015 at 07:23:30PM +0100, Hannes Frederic Sowa wrote: > > > Hello, > > > > > > On Tue, Nov 24, 2015, at 17:25, Florian Westphal wrote: >

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Hannes Frederic Sowa
On Tue, Nov 24, 2015, at 20:16, Hannes Frederic Sowa wrote: > ...also ipvs/netfilter could be used to only inspect the header and > reroute the packet to some better fitting CPU. Complete hierarchies > could be build with NUMA and addresses, packets could be rerouted into > namespaces, etc. Maybe

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Hannes Frederic Sowa
Hello, On Tue, Nov 24, 2015, at 19:59, Alexei Starovoitov wrote: > On Tue, Nov 24, 2015 at 07:23:30PM +0100, Hannes Frederic Sowa wrote: > > Hello, > > > > On Tue, Nov 24, 2015, at 17:25, Florian Westphal wrote: > > > Its a well-written document, but I don't see how moving the burden of > > > loc

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Alexei Starovoitov
On Tue, Nov 24, 2015 at 07:23:30PM +0100, Hannes Frederic Sowa wrote: > Hello, > > On Tue, Nov 24, 2015, at 17:25, Florian Westphal wrote: > > Its a well-written document, but I don't see how moving the burden of > > locking a single logical tcp connection (to prevent threads from > > reading a pa

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Hannes Frederic Sowa
Hello, On Tue, Nov 24, 2015, at 17:25, Florian Westphal wrote: > Its a well-written document, but I don't see how moving the burden of > locking a single logical tcp connection (to prevent threads from > reading a partial record) from userspace to kernel is an improvement. > > If you really have

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Rick Jones
On 11/24/2015 07:49 AM, Eric Dumazet wrote: But in the end, latencies were bigger, because the application had to copy from kernel to user (read()) the full message in one go. While if you wake up application for every incoming GRO message, we prefill cpu caches, and the last read() only has to c

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Tom Herbert
On Tue, Nov 24, 2015 at 9:16 AM, Florian Westphal wrote: > Tom Herbert wrote: >> No one is being forced to use any of this. > > Right. But it will need to be maintained. > Lets ignore ktls for the time being and focus on KCM. > > I'm currently trying to figure out how memory handling in KCM > is

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Florian Westphal
Tom Herbert wrote: > No one is being forced to use any of this. Right. But it will need to be maintained. Lets ignore ktls for the time being and focus on KCM. I'm currently trying to figure out how memory handling in KCM is supposed to work. say we have following record framing: struct recor

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Tom Herbert
On Tue, Nov 24, 2015 at 8:25 AM, Florian Westphal wrote: > David Miller wrote: >> From: Florian Westphal >> Date: Tue, 24 Nov 2015 16:27:44 +0100 >> >> > Aside from Hannes comment -- KCM seems to be tied to the TLS work, i.e. >> > I have the impression that KCM without ability to do TLS in the k

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Florian Westphal
David Miller wrote: > From: Florian Westphal > Date: Tue, 24 Nov 2015 16:27:44 +0100 > > > Aside from Hannes comment -- KCM seems to be tied to the TLS work, i.e. > > I have the impression that KCM without ability to do TLS in the kernel > > is pretty much useless for whatever use case Tom has i

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread David Miller
From: Florian Westphal Date: Tue, 24 Nov 2015 16:27:44 +0100 > Aside from Hannes comment -- KCM seems to be tied to the TLS work, i.e. > I have the impression that KCM without ability to do TLS in the kernel > is pretty much useless for whatever use case Tom has in mind. I do not get this impres

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread David Miller
From: Hannes Frederic Sowa Date: Tue, 24 Nov 2015 12:25:39 +0100 > Hello, > > David Miller writes: > >> From: Tom Herbert >> Date: Mon, 23 Nov 2015 09:33:44 -0800 >> >>> The TCP PSH flag is not defined for message delineation (neither is >>> urgent pointer). We can't change that (many people

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Eric Dumazet
On Tue, 2015-11-24 at 16:27 +0100, Florian Westphal wrote: > David Miller wrote: > > From: Tom Herbert > > Date: Mon, 23 Nov 2015 09:33:44 -0800 > > > > > The TCP PSH flag is not defined for message delineation (neither is > > > urgent pointer). We can't change that (many people have tried to ad

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Florian Westphal
David Miller wrote: > From: Tom Herbert > Date: Mon, 23 Nov 2015 09:33:44 -0800 > > > The TCP PSH flag is not defined for message delineation (neither is > > urgent pointer). We can't change that (many people have tried to add > > message semantics to TCP protocol but have always failed miserabl

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-24 Thread Hannes Frederic Sowa
Hello, David Miller writes: > From: Tom Herbert > Date: Mon, 23 Nov 2015 09:33:44 -0800 > >> The TCP PSH flag is not defined for message delineation (neither is >> urgent pointer). We can't change that (many people have tried to add >> message semantics to TCP protocol but have always failed mi

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-23 Thread Tom Herbert
On Mon, Nov 23, 2015 at 11:54 AM, David Miller wrote: > From: Tom Herbert > Date: Mon, 23 Nov 2015 09:33:44 -0800 > >> The TCP PSH flag is not defined for message delineation (neither is >> urgent pointer). We can't change that (many people have tried to add >> message semantics to TCP protocol b

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-23 Thread David Miller
From: Tom Herbert Date: Mon, 23 Nov 2015 09:33:44 -0800 > The TCP PSH flag is not defined for message delineation (neither is > urgent pointer). We can't change that (many people have tried to add > message semantics to TCP protocol but have always failed miserably). Agreed. My only gripe with

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-23 Thread Hannes Frederic Sowa
Hello Tom, On Mon, Nov 23, 2015, at 18:33, Tom Herbert wrote: > > For me this still looks a little bit like messages could be delimited by > > TCP PSH flag, where we might need to have some more fine grained control > > over and besides that just adding better fanout semantics to TCP, no? > > > Th

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-23 Thread Tom Herbert
> For me this still looks a little bit like messages could be delimited by > TCP PSH flag, where we might need to have some more fine grained control > over and besides that just adding better fanout semantics to TCP, no? > The TCP PSH flag is not defined for message delineation (neither is urgent

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-23 Thread Sowmini Varadhan
On (11/23/15 10:53), Hannes Frederic Sowa wrote: > > > > - Integration with TLS (TLS-in-kernel is a separate initiative). > > This is interesting: > > Regarding the last week's discussion about better OOB support in TCP > e.g. for SOCKET_DESTROY, do you already have a plan to handle TLS alerts

Re: [PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-23 Thread Hannes Frederic Sowa
Hello, On Fri, Nov 20, 2015, at 22:21, Tom Herbert wrote: > Kernel Connection Multiplexor (KCM) is a facility that provides a > message based interface over TCP for generic application protocols. > The motivation for this is based on the observation that although > TCP is byte stream transport pro

[PATCH net-next 0/6] kcm: Kernel Connection Multiplexor (KCM)

2015-11-20 Thread Tom Herbert
Kernel Connection Multiplexor (KCM) is a facility that provides a message based interface over TCP for generic application protocols. The motivation for this is based on the observation that although TCP is byte stream transport protocol with no concept of message boundaries, a common use case is t