Re: serializable kafka producer pool

Bump... Looking for a Kafka Producer Object pool to use in Spark Streaming inside foreachPartition On Wed, Jan 14, 2015 at 8:40 PM, Josh J wrote: > Hi, > > Does anyone have a serializable kafka producer pool that uses the > KafkaProducer.crateProducer() method? I'm trying

generate specific throughput load

Hi, Is there a way to generate a specified amount of throughput? I'm using the Stats class here to measure the throughput. Though I need to

serializable kafka producer pool

Hi, Does anyone have a serializable kafka producer pool that uses the KafkaProducer.crateProducer() method? I'm trying to use the Spark borrow feature to cache the kafka producers. Thanks, Josh

Re: Kafka producer iOS and android

t; Big Data Open Source Security LLC > http://www.stealth.ly > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > ****/ > > On Mon, Oct 20, 2014 at 1:09 PM, Josh J wrote: > > > Thanks for the tip. I would like

Re: Kafka producer iOS and android

imilar service for kafka > https://github.com/mozilla-metrics/bagheera . This project used > kafka 0.7 but you can see how its implemented. Hope that helps. > -Harsha > On Mon, Oct 20, 2014, at 08:45 AM, Josh J wrote: > > hi > > > > Is it possible to

Kafka producer iOS and android

hi Is it possible to have iOS and android to run the code needed for Kafka producers ? I want to have mobile clients connect to Kafka broker Thanks, Josh

read N items from topic

hi, How do I read N items from a topic? I also would like to do this for a consumer group, so that each consumer can specify an N number of tuples to read, and each consumer reads distinct tuples. Thanks, Josh

Re: rate of producers writing to a topic?

014 at 4:35 PM, Neil Harkins wrote: > > On Tue, Oct 7, 2014 at 3:56 PM, Josh J wrote: > >> Is there a way to monitor the rate of rates to a particular topic? I > wish > >> to monitor the frequency of incoming tuples in order to consume from the > >> topic

Re: rate of producers writing to a topic?

Interested in the total number of tuples written per millisecond per topic. On Tue, Oct 7, 2014 at 3:56 PM, Josh J wrote: > Hi, > > Is there a way to monitor the rate of rates to a particular topic? I wish > to monitor the frequency of incoming tuples in order to consume from th

rate of producers writing to a topic?

Hi, Is there a way to monitor the rate of rates to a particular topic? I wish to monitor the frequency of incoming tuples in order to consume from the topic in particular ways depending on the incoming write throughput. Thanks, Josh

kafka high level consumer - threads guaranteed to read a single partition?

Hi, For the kafka high level consumer, if I create exactly the number of threads as the number of partitions, is there a guarantee that each thread will be the only thread that reads from a particular partition? I'm following this example

Re: consumer read from specific partition

valuating the key and discarding messages that > aren’t relevant. > > The only other option I can think of if you are using the high-level API > would be finer-grained topics. > > Best Regards, > > -Jonathan > > On Aug 18, 2014, at 9:14 AM, Josh J wrote: > > >

Re: consumer read from specific partition

Is it possible to modify and use the high level consumer so that I can ignore processing certain partitions? On Mon, Aug 18, 2014 at 5:07 PM, Sharninder wrote: > On Mon, Aug 18, 2014 at 7:27 PM, Josh J wrote: > > > > You can see an example of using the Simp

Re: consumer read from specific partition

> > Implement the low level "Simple Consumer". > > > > -- > > Sharninder > > > > > > > > On Thu, Aug 14, 2014 at 2:16 PM, Josh J wrote: > > > > > Hi, > > > > > > Suppose I have N partitions. I would like to have X

consumer read from specific partition

Hi, Suppose I have N partitions. I would like to have X different consumer threads ( X < N) read from a specified set of partitions. How can I achieve this? Thanks, Josh

Re: In the middle of a leadership election, there is currently no leader...

; > > On Thu, Jul 17, 2014 at 9:01 AM, Josh J wrote: > > > Hi, > > > > I'm running three kafka brokers. I receive the following when using the > > producer > > > > panic: kafka server: In the middle of a leadership election, there is > > currentl

In the middle of a leadership election, there is currently no leader...

Hi, I'm running three kafka brokers. I receive the following when using the producer panic: kafka server: In the middle of a leadership election, there is currently no leader for this partition and hence it is unavailable for writes. Though the describe topic shows a leader for each partition.