Hi Bill, First of all, thanks for sharing the details of the project. The scale you're looking, at least in terms of total number of active sessions should be relatively easily achievable with the right hardware. In case you haven’t done it yet, I’ll only recommend that you preallocate the number of sessions and connections to the maximum you’re expecting at startup (see startup.conf session and tcp stanzas), to avoid pool reallocation delays.
Regarding the active opens, i.e, the connects, those are done from main thread because for half-opens we do not know in advance the thread they will end up on and migrating sessions and tcp connections between threads would require locks and rpcs. So, instead, we wait for peer’s syn-ack to discover the thread selected by RSS for the flow and only allocate the established connection and session at that point. In contrast, for passive opens, i.e., accepts, the syn is received on the right thread so there’s no need for extra work. If you do something similar to what session_mq_connect_handler does, you can rpc from your workers the main thread and in theory even get some coalescing benefits as the connect requests accumulate in a queue. Did you try that and hit performance limitations? Regards, Florin > On Feb 5, 2021, at 6:14 AM, Bill Vaughan <billvaug...@gmail.com> wrote: > > Hi all, > > I have a need to call vnet_connect() from worker threads but the stack > requires it to be called from the main thread. For some context, the project > I am working on includes a TCP PEP (Performance Enhancing Proxy). A PEP can > speed up TCP connections over high latency paths, such as satellites. The > PEP terminates the connection near the client, transports the data over the > high latency path using a proprietary protocol, then reestablishes the TCP > connection to the server on the other end. This greatly increases TCP ramp-up > over high RTTs. My VPP application runs on both sides of the high latency > path (ie: client and server model). > > The PEP is designed to process tens of thousands of simultaneous TCP > connections at a rate of many 100 per second. So to achieve this I wanted to > distribute the TCP flows across several workers but keep each individual flow > on the same worker. Currently I have to context switch to the main thread to > perform the connect, but all other operations can be performed within the > worker context (accept(), svm fifo read/write). > > I went down the path to try to enable vnet_connect() on workers but there are > asserts() at several places in this path. > > Any insight or work-around for this would be appreciated. I am using VPP > 2009. > > Thanks, > -Bill > > > > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#18681): https://lists.fd.io/g/vpp-dev/message/18681 Mute This Topic: https://lists.fd.io/mt/80406012/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
