The big syslog daemons support Kafka since a while back. rsyslog: http://www.rsyslog.com/doc/master/configuration/modules/omkafka.html
syslog-ng: https://czanik.blogs.balabit.com/2015/01/syslog-ng-kafka-destination-support/#more-1013 And Bruce might be of interest aswell: https://github.com/tagged/bruce On the less daemony and more tooly side of things are: https://github.com/fsaintjacques/tail-kafka https://github.com/mguindin/tail-kafka https://github.com/edenhill/kafkacat 2015-01-28 19:47 GMT+01:00 Gwen Shapira <gshap...@cloudera.com>: > It sounds like you are describing Flume, with SpoolingDirectory source > (or exec source running tail) and Kafka channel. > > On Wed, Jan 28, 2015 at 10:39 AM, Fernando O. <fot...@gmail.com> wrote: > > Hi all, > > I'm evaluating using Kafka. > > > > I liked this thing of Facebook scribe that you log to your own machine > and > > then there's a separate process that forwards messages to the central > > logger. > > > > With Kafka it seems that I have to embed the publisher in my app, and > deal > > with any communication problem managing that on the producer side. > > > > I googled quite a bit trying to find a project that would basically use > > daemon that parses a log file and send the lines to the Kafka cluster > > (something like a tail file.log but instead of redirecting the output to > > the console: send it to kafka) > > > > Does anyone knows about something like that? > > > > > > Thanks! > > Fernando. >
