If one is looking at log aggregation/reporting/alerting systems, I strongly suggest looking at Graylog. It is (like ELK) built on top of Easticsearch, but it handles the Elastic management for you. It has a very polished interface and nice dashboard / alerting features. We’re using it internally for ~150 devices logging to one Graylog VM. It is also very scalable for larger installations and/or high availability.
The Graylog folks have pre-rolled VM images which would let you get going quickly and use it for parsing these existing files. They also have OS packages, docker image, etc. -Pete From: <tech-boun...@lists.lopsa.org> on behalf of Graham Dunn <g...@kurai.org> Date: Thursday, March 31, 2016 at 06:43 To: Simon Lyall <si...@darkmere.gen.nz> Cc: LOPSA Tech List <t...@lopsa.org> Subject: Re: [lopsa-tech] Tool for searching and browsing log files. I've found a small ELK stack reasonable (Kibana is pretty useful for finding stuff), but needs to be monitored lots (ie, logstash will stop working for no reason, same with elasticsearch). TBH, splunk is very very good at this and easy to set up. It *can* be expensive, but if it's worth money to you to know about things, I'd look into it as well. On Thu, Mar 31, 2016 at 6:46 AM, Simon Lyall <si...@darkmere.gen.nz> wrote: I'm looking for a tool that might handle this nicely. I have some asterisk log files that are generated by daemontools' multilog. The problem is that daemontools rotates logs every few minutes at the volumes I do so a single call can be scattered across several files. Total files might be a few GB. Does anyone know a good tool that let us search across multiple files for strings and display them? (along with nearby lines). The basic requirements are to trace a problem call by finding a number or call id and jumping around checking the 10 lines or so on each side of it. What I've looked at so far: * The team currently use vi which doesn't support multiple files well * I had a look at use multiple search in less but this was pretty slow and didn't seem to work nicely. * logstash and Elasticsearch would probably be the longer team option although there doesn't seem to be a good built-in asterisk filter for grok. * lnav ( http://lnav.org/ ) looks nice but doesn't have support for multilog format and doesn't appear to actually search multiple generations of files. -- Simon Lyall | Very Busy | Web: http://www.simonlyall.com/ "To stay awake all night adds a day to your life" - Stilgar _______________________________________________ Tech mailing list Tech@lists.lopsa.org https://lists.lopsa.org/cgi-bin/mailman/listinfo/tech This list provided by the League of Professional System Administrators http://lopsa.org/ _______________________________________________ Tech mailing list Tech@lists.lopsa.org https://lists.lopsa.org/cgi-bin/mailman/listinfo/tech This list provided by the League of Professional System Administrators http://lopsa.org/
_______________________________________________ Tech mailing list Tech@lists.lopsa.org https://lists.lopsa.org/cgi-bin/mailman/listinfo/tech This list provided by the League of Professional System Administrators http://lopsa.org/
