Mike Sheinberg wrote on 1/1/2015 11:00 PM: > For the background, I'm using logstash as a netflow collector --> ES. I was > previously using the dns filter of logstash to reverse lookup IP fields in > realtime but that caused performance issues and it seems like records were > being lost. So my question is - is it more efficient for me to continue > trying to tackle this in logstash (before records are placed into ES) or > would it make more sense for me to do something after the record is in ES? > I don't have an issue with the delay of having the DNS resolution, so I > imagine going through the previous hour, every hour to batch update records.
I've found that running a caching nameserver on the logstash server and setting /etc/resolv.conf to use the local name server massively improves the performance of the dns filter in logstash. Otherwise, you lots of off-server dns lookups which take time. --[Lance] -- GPG Fingerprint: 409B A409 A38D 92BF 15D9 6EEE 9A82 F2AC 69AC 07B9 CACert.org Assurer -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/54A61D01.7000700%40bearcircle.net. For more options, visit https://groups.google.com/d/optout.
