Thank you for answer Michael. Actually, I have made a simple producer from Pcap to Kafka. Since it is not structured, so it is difficult for further processing by a consumer. But, I will take a look at Avro as you mentioned.
I just wondering, if there are any proper implementation for this requirement, because I couldn't find any tool in the kafka ecosystem page. https://cwiki.apache.org/confluence/display/KAFKA/Ecosystem Best Regards, Aris. On Mon, Oct 10, 2016 at 6:55 PM, Michael Noll <mich...@confluent.io> wrote: > Aris, > > even today you can already use Kafka to deliver Netflow/Pcap/etc. messages, > and people are already using it for that (I did that in previous projects > of mine, too). > > Simply encode your Pcap/... messages appropriately (I'd recommend to take a > look at Avro, which allows you to structure your data similar to e.g. > Pcap's native format [1]), and then write the encoded messages to Kafka. > Your downstream applications can then read the encoded messages back from > Kafka, decode, and commence processing. > > That was a brief summary to get you started, feel free to take a look at > the Apache Kafka docs at kafka.apache.org and/or ask further questions > here. > > -Michael > > > > > [1] https://wiki.wireshark.org/Development/LibpcapFileFormat > > On Mon, Oct 10, 2016 at 11:19 AM, Aris Risdianto <a...@smartx.kr> wrote: > > > Hello, > > > > > > Is there any plan or implementation to use Kafka for delivering > > sFlow/NetFlow/Pcap messages? > > > > > > Best Regards, > > Aris. > > >
