Interesting. It appears that we are dropping the reenable action from
your thread sometimes. I'm guessing there is an issue with the
readReschedule.
I'll give this a go in my transparent dev environment. A couple things
you might want to look at.
* Brian uncovered a fairly serious SSL startup issue tracked in
TS-3424. It is now fixed on main and is (or will be soon) fixed on 5.2.x.
* If your read event got the trigger cleared, the readReschedule
wouldn't really do anything and things might stall. I saw odd things
with regards to SSL and reschedules which I've reported in TS-3453.
There is a proposed fix in there you might want to give a go. Not at
all clear that is what you are running into. I'm still getting my head
around the scheduling logic. You might check whether the
this->read.triggered and this->read.enabled bits are set in
SSLNetVConnection::reenable before you call readReschedule.
In any case, could you file a bug for this. Definitely an issue we need
to track and resolve.
On 3/19/2015 8:41 AM, Lev Stipakov wrote:
Hello,
I made a simple plugin that sets up TS_SSL_SNI_HOOK and creates a
blind tunnel from a separate thread. With low load everything works
fine, but with moderate load (100 simultaneous users, each user sends
200 HTTPS requests) I see somewhat strange behavior.
On a client side I use Tsung, which creates users and sends number of
requests per user. For each user Tsung waits for a response before
sending a new request, so if response never arrives, a particular user
(and the whole test) stalls.
So, with load mentioned above I see few 'stalled' connections on both
client and proxy – netstat shows them as ”established”, ATS seems to
have data structures for those (checked
proxy.process.net.connections_currently_open value), but no traffic
goes between proxy and client.
Client side (.175):
tcp 0 0 10.133.3.175:40737 10.133.3.250:443 ESTABLISHED 14332/beam.smp
(more similar connections here)
Proxy side (.250 is a server):
tcp 0 0 10.133.3.250:443 10.133.3.175:40737 ESTABLISHED 28117/traffic_serve
(more similar connections here)
I checked traffic.out log and found out that
”SSLNextProtocolAccept:mainEvent” does not get called as many times as
it should. This can probably be explained by the fact that client does
not send requests for given user anymore if response to previous
request hasn't been received. Which, in turn, may indicate that at
some point tunnel has not been created.
The interesting thing is that everything works fine if a tunnel is
created directly from TS_SSL_SNI_HOOK but not from the separate
thread.
The plugin code is very simple – I set up TS_SSL_SNI_HOOK and start a
thread with TSThreadCreate. When hook got called, I push TSVConn to a
thread-safe queue. The thread wakes up when item has been pushed,
calls TSVConnTunnel / TSVConnReenable for given vconn and then waits
for the next item. I have attached the code.
Do you guys have any suggestions how I can investigate this problem further?
