On Sun, 28 Feb 2010, Doug Hughes wrote: > Paul DiSciascio wrote: >> I'm looking for a good way to share log files on a centralized syslog >> server with about 10-20 people/developers who are familiar with the log >> formats but not very much with unix tools. They want an easy way to >> dig thru the logs and filter out junk they're not interested in, but >> still have near realtime visibility. Obviously, splunk can do this, >> but it's pricey and their documentation seems to indicate that 20 >> concurrent users would be a lot to ask for without a lot of hardware. >> I really only need an interface capable of some rudimentary filtering, >> and if possible the ability to save those searches or filters. Does >> anyone have any suggestions short of writing this myself? >> >> > You might be interested in SEC (simple event correlator) for this > purpose. But, if you just want a presentation interface, logsurfer might > be more what you are looking for. SEC is much more like splunk while > logsurfer is more of a realtime filtering monitor.
I'm not sure what you have seen of splunk, but it and SEC have very little in common. splunk allows for arbatrary search queries against your past log data (and indexes it like crazy to make the search fairly efficiant) SEC watches for patterns (or combinations of patterns) to appear in the logs and generates alerts. splunk can simulate SEC's functionality by doing repeated queries against the logs, but that's fairly inefficant. the answer to the original question, it depends a lot on the amount of data that you are working with. If you can fit it all in ram on a machine, then there are a lot of things that you can use to query it. The problem comes when you can no longer fit it in ram and have to go to disk, at that point you need an application that does a lot of indexing (and/or spreads the load across multiple machines, depending on how much data you have and how fast you want your answers) you say that your users are not familiar with unix tools, are they familiar with using SQL for queries? David Lang _______________________________________________ Discuss mailing list Discuss@lopsa.org http://lopsa.org/cgi-bin/mailman/listinfo/discuss This list provided by the League of Professional System Administrators http://lopsa.org/
