John Young wrote: > > Declan's article ran on Friday July 21 day and the hits from it did not > seem to affect the sites. Saturday, an AP story appeared but it did not > include links to the site, however, Drudge Report picked up the AP story > and provided a munged link to jya.com: > > http://jya.com/crypto.htmhttp://jya.com/crypto.htm Is the machine a unix machine? if so, just do this: grep "http://jya.com/crypto.htmhttp://jya.com/crypto.htm" access_log | wc -l Where access log is the logfile. This will give you the number of "hits" on this bad url. > The error log has jumped from 13MB to only 15MB since July 21. (By far the > largest cause of previous errors is the pernicious "favicon.ico.") That's just Microsoft stupidity. It looks for a favorite icon to add to the list of favorites. For fun, get an pic of Janet Reno, replace the nose with a penis, and store it as this file. :) > Soon after the Drudge attack began, this entry in the error log started to > appear and repeated every few minutes, sometimes every minute (entries > numbered by us for reference): > > (1) (32)Broken pipe: accept: (client socket) > > This entry had appeared only infrequently previously. This means a time out between the server and the client - or unlikely, a Denial of Service. Usually, it means someone couldn't get the entire .htm file from you. > Several hours later entry (2) appeared dozens of times at the > same clock time: > > (2) [warn] child process 736 still did not exit, sending a SIGTERM > > Followed by several iterations of entry (3) at the same clock time: > > (3) [error] child process 628 still did not exit, sending a SIGKILL No clue what this is. From the looks of it, it's trying to spawn off more servers, but it can't because the process table may be full. What happens when a request comes in, is that the web server "forks" or makes a copy of itself in memory to handle the request, if it can't build any more of these it can't handle the requests. (It doesn't actually duplicate itself per say, just the stack and variables, etc.) > And then: > > (4) Site site1 has invalid certificate: 4999 Certificate files do not exist. > (5) Site site2 has invalid certificate: 4999 Certificate files do not exist. Uh, this looks like you were getting SSL requests, but what the fuck? > (6) [crit] (98)Address already in use: make_sock: could not bind to port 80 Now it looks like someone - either a monitoring script, or a sysadmin tried to rerun the server after the crash, but since a copy of it - or something was running off port 80, it couldn't start up. > (7) [notice] caught SIGTERM, shutting down This just means someone or something told that process to die. Sig Term means commit suicide. :) > (8) Site site1 has invalid certificate: 4999 Certificate files do not exist. > (9) Site site2 has invalid certificate: 4999 Certificate files do not exist. Ditto. > (10) [notice] Apache/1.3.6 (Unix) mod_perl/1.21 mod_ssl/2.2.8 OpenSSL/0.9.2b > configured -- resuming normal operations Ah, ok, so you were running SSL. BTW: the above is a little bit dated. Apache is at least 1.3.12, with 1.4 still being worked on... You'll likely want them to upgrade. > The pattern of these series of entries continues, with shutdowns and restarts > repeating since Saturday, July 22. > > During the outage period we have been sent frequent automatic messages like > the following: > > (11) Over the past fifteen minutes, the CPU has been heavily loaded. > > This will result in noticible performace loss. Consider moving some > of the > services to other Cobalt servers, or reduce the complexity of the CGI > scripts running on the Cobalt server itself. I hope you're not using Cobalt Qube servers - I've played with those about two years ago, and they were insecure as all hell. Anyhow, this message is simply telling you that the CPU is heavily used. Likely whomever configured Apache didn't set (reasonable) limits on the number of children (max connections) and so when the load too off, the box started thrashing (swapping ram to disk, with disk being at least 1000 times slower than RAM.) Bottom line: whomever your sysadmin is didn't tune the box and apache properly, and is running an old version. > 1 minute load average: 27.79 > 5 minute load average: 68.67 > 15 minute load average: 84.27 > > (12) Memory on the Cobalt server is heavily used. > The Cobalt server needs more memory than it currently has. > > Consider adding more DRAM to the server. > > Total memory is: 162376 KB > Used memory is: 161012 KB > Free memory is: 1364 KB > Percent used is: 99 > > (13) Your server (cob487) is not responding on the port (80) we are > monitoring - > please let us know if this is going to be a permanent condition. > > If you have a support contract with us, and this is within normal > business > hours, feel free to send an e-mail to [EMAIL PROTECTED] or [EMAIL PROTECTED] > regarding the problem you are having. > > If you are doing work on your server, you can reply to this message > and it will > be noted by our SOC staff. If this is an unexpected problem for you, > you may > wish to contact anyone else at your company who might be working on the > server to find out if they are aware of the situation. > > This ticket will remain open until the server is back online, is > accepting > connections, or you are notified by our SOC staff that the port we are > monitoring has been changed to avoid alarms on our end. > > Please let us know if you need anything. > > Thanks, > > --Server Operations Center > > We would appreciate advice on whether these log entries and messages are > consistent with simple overloading or could indicate an attack, even a > presumbably accidental attack by Drudge (who has still not answered my > Saturday e-mail to correct the URL). > > Thanks very much. -- ----------------------Kaos-Keraunos-Kybernetos--------------------------- + ^ + :Surveillance cameras|Passwords are like underwear. You don't /|\ \|/ :aren't security. A |share them, you don't hang them on your/\|/\ <--*-->:camera won't stop a |monitor, or under your keyboard, you \/|\/ /|\ :masked killer, but |don't email them, or put them on a web \|/ + v + :will violate privacy|site, and you must change them very often. [EMAIL PROTECTED] http://www.sunder.net ------------
