Hello,
To sum this one up from my end, my suspicion is that the TCP stack
buffer
overflow that lead to the limited TCP stack zombie state was because I
had
a TCP server process that accept()ed every incoming TCP connection
without
an internal connection overload mechanism,
at the same time as I had a third party flooding.
Based on this, my working conclusion is that this accept():ing lead to
the
TCP stack run out of mbuf:s (without any warning left in the the dmesg
about it), and through some more or less exotic logic in the TCP stack,
the
'hangup state' as explored by Karlis and myself in the previous ML
posts
was entered into.
Since this 'hangup state' was experienced, the TCP server process has
been
enabled with an overload mechanism and I have not seen any 'hangup
state'
since, which supports the hypothesis above.
So that about it from my end.
As for my "solution" to this issue - there were multiple websites on
server that were generating random URLs every time the page was visited,
something like this:
http://domain.lv/index.php?2fse%3Djl6Vs1FqHZ6zgZzcUWH+1O6V5iO6zHf%3DjzfSUnVBn6
http://domain.lv/index.php?5fZ9%3D5fBnV%3DfZ80iwg9CicpBH2fse%3DW2c88
Needless to say, web crawlers were scanning these webpages 24/7 and
creating quite heavy load for the webserver. As soon as we added
robots.txt for these specific webpages and later redesigned them with
static URLs, issue went away. Something seems to trigger this bug on
OpenBSD, and exact same server configuration was working just fine up
until upgrade to OpenBSD 5.2.
In my case webpages were served by OpenBSD Apache + PHP 5.3.21 (with
curl, gd, imap, mcryp, mysql, mysqli and pdo_mysql extensions installed
from packages).
--
Karlis
