This may be because the 0.8 producer sticks to a partition during metadata refresh intervals. You can get around that by specifying a key:
--property parse.key=true --property key.separator=### Each line would then be: KEY###MESSAGE The key is used for partitioning but will also be stored with the message but that shouldn't break your consumers if they extract just the message. We have a partition key which is thrown away after partitioning but that has not been integrated with the console producer. Thanks, Joel On Wed, Apr 09, 2014 at 07:13:16PM -0400, Ian Friedman wrote: > Hey guys. We recently migrated our production cluster from 0.7.2 to 0.8.1. > One of the tools in the 0.7 distribution was something called Producer Shell, > which we used in some cron jobs to do some manual addition of messages (for > messages that got dropped for various reasons). So in 0.8 that is gone, and I > have tried replacing it with the console Producer. It seems like the console > producer only produces to a single partition whereas the producer shell used > to randomly distribute the messages. Is this behavior configurable? AM I > missing something obvious? > > Thanks in advance > --Ian
