This kind of situation is exactly what tools like Splunk and VMware's
Log Insight were designed for. Tools like these offer both automatic
log parsing/analysis and a nice front-end for manual searching. The
stack of logstash, elasticsearch, and kibana (and watcher) all from
elastic.co might b
On Mon, 24 Aug 2015, Page, Jeremy wrote:
Sorry, was not saying don't look at logs, just saying logs are only reactive
and only see things you're logging (if the server crashes you may log nada but
that's definitely an issue! I also personally find correlation easier when I
have graphic data bu
____________
> > From: tech-boun...@lists.lopsa.org [tech-boun...@lists.lopsa.org] on
> behalf of Edward Ned Harvey (lopser) [lop...@nedharvey.com]
> > Sent: Monday, August 24, 2015 6:51 AM
> > To: Adam Moskowitz; tech@lists.lopsa.org
> > Subject: Re: [lo
David Lang [da...@lang.hm]
Sent: Monday, August 24, 2015 9:03 AM
To: Page, Jeremy
Cc: Edward Ned Harvey (lopser); Adam Moskowitz; tech@lists.lopsa.org
Subject: Re: [lopsa-tech] Server Overload and Log Processing
On Mon, 24 Aug 2015, Page, Jeremy wrote:
> I think the problem is looking at it a
ld) be
>> the thing you are trying to verify.
>>
>>
>>
>> From: tech-boun...@lists.lopsa.org [tech-boun...@lists.lopsa.org] on
>> behalf of Edward Ned Harvey (lopser) [lop...@nedharvey.com]
>> Sent: Monday, August 2
:51 AM
To: Adam Moskowitz; tech@lists.lopsa.org
Subject: Re: [lopsa-tech] Server Overload and Log Processing
From: tech-boun...@lists.lopsa.org [mailto:tech-boun...@lists.lopsa.org]
On Behalf Of Adam Moskowitz
I don't see how that can be true: If "a bunch of users" will get errors,
[lop...@nedharvey.com]
Sent: Monday, August 24, 2015 6:51 AM
To: Adam Moskowitz; tech@lists.lopsa.org
Subject: Re: [lopsa-tech] Server Overload and Log Processing
> From: tech-boun...@lists.lopsa.org [mailto:tech-boun...@lists.lopsa.org]
> On Behalf Of Adam Moskowitz
>
> I don'
hrmmm well .
There's a good list here:
http://baudlabs.com/top-free-and-open-source-log-management-software/
logwatch has two problems: Although it does a good job in terms of having
predefined rules to identify undesirable log entries, it only runs according to
cron schedule, and it g
> From: tech-boun...@lists.lopsa.org [mailto:tech-boun...@lists.lopsa.org]
> On Behalf Of Adam Moskowitz
>
> I don't see how that can be true: If "a bunch of users" will get errors,
> I believe your page download tester will also see those same errors. If
> it's not seeing those errors, what good
On Sun, 23 Aug 2015, Hans van der Made wrote:
Yes, you should still be looking at your logs, but I believe that what's
more critical is that you monitor the service *from the user's point of
view*, and that monitoring should reflect the users' experiences as
closely as possible. If you do tha
> Yes, you should still be looking at your logs, but I believe that what's
> more critical is that you monitor the service *from the user's point of
> view*, and that monitoring should reflect the users' experiences as
> closely as possible. If you do that, you'll know when things are wrong
> with
Edward Ned Harvey (lopser) wrote:
> a bunch of users will get "page cannot be displayed" and you won't know
> about it
> . . .
> We have systems that periodically (every minute) download pages from
> the server, and alert us if they don't get the expected results.
> . . .
> But if we've configured
> From: Graham Dunn [mailto:g...@kurai.org]
>
> The Linux logwatch package operates on a "these patterns are okay, these
> patterns are bad, anything else is unmatched, here's those ones" basis.
That's definitely an improvement, thanks. I got a 10 page email last night,
hopefully it won't be 10
> From: Graham Dunn [mailto:g...@kurai.org]
>
> The Linux logwatch package operates on a "these patterns are okay, these
> patterns are bad, anything else is unmatched, here's those ones" basis.
That's definitely an improvement, thanks. I got a 10 page email last night,
hopefully it won't be 10
There are lots of tools out there for watching logs and alerting on specific
patterns.
I believe that logwatch just looks at the logs a line at a time, not trying to
keep context
Simple Event correlator can match on individual lines, but can also keep context
so that it can alert on combinat
The Linux logwatch package operates on a "these patterns are okay, these
patterns are bad, anything else is unmatched, here's those ones" basis.
There are many modules for different daemons. It might be a good starting
point.
On Sat, Aug 22, 2015 at 10:16 AM Edward Ned Harvey (lopser) <
lop...@ned
I am surprised nobody had a "just use this product" or "just google for this
search term" response -
Let me describe a little more what I'm looking for -
So you create a VM, and turn on apache. Of course it has a default config file,
including a default number of MPM preforks and threads and
On Fri, 21 Aug 2015, Edward Ned Harvey (lopser) wrote:
I want to know if a web server gets overrun by too much traffic requests. I
certainly know how to monitor memory, and tweak the MPM and stuff in apache
config files, but I assume if it runs out of threads or memory or anything, it
will thr
Personally, I'd look into parsing the mod_status output first. Access logs
tend to grow quickly, so that's a lot of reading.
logstash might help you make sense of all kinds of logs.
Best,
Hans
NL
On Fri, Aug 21, 2015 at 3:10 PM, Edward Ned Harvey (lopser) <
lop...@nedharvey.com> wrote:
> I wan
There's something else you should consider at the same time. There are
many ways that you can setup connection throttling such that the web
server is prevented from being overrun in some of the more obvious ways
that you might not want. There are plugin modules for bandwidth
limiting, but you c
Edward Ned Harvey (lopser) wrote:
> I want to know if a web server gets overrun by too much traffic requests.
First you need to define "overrun." Not for the mailing list, but for
yourself.
> I'm looking for something that knows how to process the logs of . . .
Assuming you want to know of the p
I want to know if a web server gets overrun by too much traffic requests. I
certainly know how to monitor memory, and tweak the MPM and stuff in apache
config files, but I assume if it runs out of threads or memory or anything, it
will throw errors into the log file, which are immediately buried
22 matches
Mail list logo
