Re: consumer read from specific partition

The high level API gives you access to the raw Kafka event which is MessageAndMetadata, which has two pieces: key and actual message. Best Regards, -Jonathan On Aug 18, 2014, at 9:50 AM, Josh J wrote: >> One tactic that might be worth exploring is to rely on the message key to > facilitate t

Re: consumer read from specific partition

> One tactic that might be worth exploring is to rely on the message key to facilitate this. > It would require engineering careful functions for the key which hashes to the partitions for your topic(s). It would also mean that your consumers for the topic would be evaluating the key and discardin

Re: consumer read from specific partition

One tactic that might be worth exploring is to rely on the message key to facilitate this. It would require engineering careful functions for the key which hashes to the partitions for your topic(s). It would also mean that your consumers for the topic would be evaluating the key and discarding

Re: consumer read from specific partition

Is it possible to modify and use the high level consumer so that I can ignore processing certain partitions? On Mon, Aug 18, 2014 at 5:07 PM, Sharninder wrote: > On Mon, Aug 18, 2014 at 7:27 PM, Josh J wrote: > > > > You can see an example of using the SimpleConsumer here > > < > > > > > https

Re: consumer read from specific partition

On Mon, Aug 18, 2014 at 7:27 PM, Josh J wrote: > > You can see an example of using the SimpleConsumer here > < > > https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example > > > > Any suggestions on where in the code to modify the high level producer to > support reading fro

Re: consumer read from specific partition

> You can see an example of using the SimpleConsumer here < https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example > Any suggestions on where in the code to modify the high level producer to support reading from specific partitions ? Thanks, Josh . On Thu, Aug 14, 2014 at

Re: consumer read from specific partition

You can see an example of using the SimpleConsumer here . On Thu, Aug 14, 2014 at 3:23 AM, Sharninder wrote: > Implement the low level "Simple Consumer". > > -- > Sharninder > > > > On Thu, Aug 14, 2014 at 2:16 PM

Re: consumer read from specific partition

Implement the low level "Simple Consumer". -- Sharninder On Thu, Aug 14, 2014 at 2:16 PM, Josh J wrote: > Hi, > > Suppose I have N partitions. I would like to have X different consumer > threads ( X < N) read from a specified set of partitions. How can I achieve > this? > > Thanks, > > Josh >

consumer read from specific partition

Hi, Suppose I have N partitions. I would like to have X different consumer threads ( X < N) read from a specified set of partitions. How can I achieve this? Thanks, Josh