Re: console producer question

2014-04-15 Thread Joe Stein
Thanks Magnus, very cool I added it to the client page https://cwiki.apache.org/confluence/display/KAFKA/Clients cheers! /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop

Re: console producer question

2014-04-10 Thread Ian Friedman
Magnus this worked in our scripts perfectly, thanks a bunch!! --Ian On Apr 9, 2014, at 8:39 PM, Magnus Edenhill wrote: > Hey Ian, > > this is where a tool like kafkacat comes in handy, it will use a random > partitioner by default (without the need for defining a key): > > tail -f /my/log |

Re: console producer question

2014-04-09 Thread Ian Friedman
Thanks Magnus! I will definitely check this out —Ian On Apr 9, 2014, at 8:39 PM, Magnus Edenhill wrote: > Hey Ian, > > this is where a tool like kafkacat comes in handy, it will use a random > partitioner by default (without the need for defining a key): > > tail -f /my/log | kafkacat -b myb

Re: console producer question

2014-04-09 Thread Magnus Edenhill
Hey Ian, this is where a tool like kafkacat comes in handy, it will use a random partitioner by default (without the need for defining a key): tail -f /my/log | kafkacat -b mybroker -t mytopic See https://github.com/edenhill/kafkacat 2014-04-10 6:13 GMT+07:00 Ian Friedman : > Hey guys. We

Re: console producer question

2014-04-09 Thread Joel Koshy
This may be because the 0.8 producer sticks to a partition during metadata refresh intervals. You can get around that by specifying a key: --property parse.key=true --property key.separator=### Each line would then be: KEY###MESSAGE The key is used for partitioning but will also be stored with