On 11/28/12 5:56 AM, Aidan McGurn wrote:
Hi,
We are looking at mitigating our system against any issues associated with long
term up time of ATS/ATM processes e.g. if any gradual memory growth.
To this end we want to close the ATS process to refresh it say every 2 days
(this time was used on Apache2 on previous project) -
Options considered:
1. While TS shuts down, TM buffers the connections - I don't think this
is practical or is it possible? (could be under 50K TPS)
or
2. We redirect the traffic away from this TS node and then issue:
/traffic_line -S //stop local node TS
./traffic_line -U //start local node TS
Here we would have to wait for the system to drain, if there's a recommended
way to check this?
I personally favor #2, which is what I did (and would do) on prod boxes.
In addition to this, when you take the system out of rotation, you want to
start sending a Connection: close back to all clients. I wrote a plugin that
could do this, Yahoo open sourced it recently but I don't know if it works
in ATS (only one way to find out). The plugin is called header_rewrite, and
the config to accomplish this is
cond %{READ_REQUEST_HDR_HOOK} [AND]
cond %{ACCESS:/url/local/www/html/status.html} [NOT]
add-header Connection "close"
What this says is, if the file .../status.html doesn't exist, start sending
a Connection: close header.
Implementing this particular use case in a custom C/C++ or Lua script would
be trivial, and you can change the logic to check for some other condition
(e.g. a value in a shm). You can here also add a condition such that you
close the connections after some number of requests, or randomly(),
preventing a connection from being used/abused forever (I think there's
examples for that).
I hope that helps.
-- Leif
