Hello Mayur, I'm not sure I fully understand your questions, but here's some information that might help:
If you have control over how logs are written, then I suggest you write them in JSON and use rsyslog to parse them automatically. Here's a blog post that show you how you can do that: http://blog.sematext.com/2013/05/28/structured-logging-with-rsyslog-and-elasticsearch/ If you can't control that and you have to parse "unstructured" logs, I see 3 options: - use mmnormalize. I didn't use it yet, but maybe you can get help from the mailing list if you need it. Here are a few links: http://www.rsyslog.com/doc/mmnormalize.html http://www.rsyslog.com/using-rsyslog-mmnormalize-module-effectively-with-adiscon-loganalyzer/ - pay Adiscon to make a custom parser for you (it should be rather cheap and fast): http://www.rsyslog.com/professional-services/custom-development/ - use some third party tool, such as Logstash, to do the parsing for you: http://logstash.net/ On the storing and analysis side, I think Elasticsearch is a good option because of its speed, scalability and tooling around it. Here's a link on how you can do that, from rsyslog all the way to the UI: http://blog.sematext.com/2013/07/01/recipe-rsyslog-elasticsearch-kibana/ And here are a couple of talks about using Elasticsearch for logging that you might find useful. This one is mine and is more about the way you send logs to ES and the tooling around it: https://www.youtube.com/watch?v=yDLtyLi6Ny8 This one is from the guy that made Kibana and is more about how you can analyze your logs with Kibana: http://www.elasticsearch.org/webinars/using-elasticsearch-for-log-search-and-analysis/ If you need anything on the Elasticsearch side, like support, custom development or consulting, you can contact the company I work for: http://sematext.com/services/index.html If you need any free help, please use the mailing list of the software you want to talk about. This way, other people can benefit from our conversation. Thanks for your understanding! > Hello Sir, > > Is there any web service or coding part possible for rsyslog that > will give access to log parsing and analysis ?? > > In the sense that will able to parse logs on one machine and give > analysis in the form of web page or interactive GUI. > > I am asking as I have told by my mentor to write some code regarding > the log analysis I also know > > there is no need but I have to write ;) It should possible to write within 10-15 days. > > Or any other coding task !! > > Please comment on this or suggest some idea !! > > Seeking for the guidance, > > Thanks !! -- *Cheers, Mayur*. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
