On Fri, 14 Jun 2013, Radu Gheorghe wrote:
Hi Mahesh,
If you don't need mysql for a specific reason, I'd suggest you try thowing
your logs in Elasticsearch. Here's a tutorial:
http://wiki.rsyslog.com/index.php/HOWTO:_rsyslog_%2B_elasticsearch
I assume you'll get way better insert and query performance than you can
with mysql (ie: with bulks, I get 10-20K logs indexed per second on my $500
laptop. Then I can query in 100M-200M logs within a second. Depends on your
settings). Plus, it's super-easy to scale Elasticsearch by adding new nodes.
For querying, there are several, tools, the most popular being Kibana:
http://three.kibana.org/
Just to note, one of the things that makes MySQL so slow or Mahesh is it's
safety features. After each insert, MySQL makes sure the data is safe on disk
before it considers the insert complete. If the system crashes, the data will be
there. There are config options to override this in MySQL.
To get the numbers that elasticsearch is getting on your laptop, it's almost
certinly not doing this.
this is probably acceptable, but you do need to be aware of the tradeoff.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
