> From: tech-boun...@lists.lopsa.org [mailto:tech-boun...@lists.lopsa.org]
> On Behalf Of Edward Ned Harvey (lopser)
> 
> modems, and taking 10 minutes to start apache.  That *is* a big F-U, or just a
> shitty service.

FWIW, the service degradation to the point of being merely shitty service, has 
been several months.

But the service degradation to the point of constantly falling over and 
triggering all the alarms hundreds of times per day, started on the first 
weekend of Sept, and was *mostly* confined to weekends in september (hundreds 
of alarms per day on weekends, and merely 1-2 dozen alarms per day on 
weekdays).  Which is suggestive that the problem was related to Amazon's 
rolling reboots responding to the xen bug.

The xen bug patching was *supposed* to be completed by Oct 1st, but every day 
from Oct 1 to Oct 5 was just like the weekend in Sept.  Which is suggestive 
that they missed their deadline.  And the last alarm was triggered Oct 6th.  So 
we now have 4 days continuous operation without alarms.

We're going to wait through one weekend and if no more alarms, re-enable the 
email alerts on Monday.

All of this is suggestive that the cripplingly broken, not merely just bad 
performance problem, was probably related to the xen bug rolling reboots.  And 
it's probably over now.  Merely lasted a month.

8 seconds to "ls" a directory with 76 items in it - that was today.  Back to 
merely shitty service.  No more alarms, just a really slow server doing its job 
correctly.
_______________________________________________
Tech mailing list
Tech@lists.lopsa.org
https://lists.lopsa.org/cgi-bin/mailman/listinfo/tech
This list provided by the League of Professional System Administrators
 http://lopsa.org/

Reply via email to