I am not familiar with logstash, but in custom log replay tool (used to
replay messages logged locally in case if e.g. kafka was not available and
useful in some other scenarios) I've seen it reaching 30,000 messages/sec
with avg message size of 4.5 kilobytes, all with regular production load on
ka
Hi Gwen,
Well I have gone through this link while trying to setup my Logstash Kafka
handler,
https://github.com/joekiller/logstash-kafka
I could achieve what I was looking for but the performance is badly
affected while trying to write a big file of GB's.
I guess there should be some way so as t
I'm wondering how much of the time is spent by Logstash reading and
processing the log vs. time spent sending data to Kafka. Also, I'm not
familiar with log.stash internals, perhaps it can be tuned to send the data
to Kafka in larger batches?
At the moment its difficult to tell where is the slowdo
Hi All,
I am having some log files of around 30GB, I am trying to event process
these logs by pushing them to Kafka. I could clearly see the throughput
achieved while publishing these event to Kafka is quiet slow.
So as mentioned for the single log file of 30GB, the Logstash is
continuously emitt
