RE: Experiences testing new producer performance across multiple threads/producer counts

2015-05-17 Thread Garry Turkington
or not: > > Select-rate > > Io-wait-time-ns-avg > > Io-time-ns-avg > > > > 3. Batch size matters, so take a look at the sensor batch-size-avg > > and > see > > if the average batch size makes sense or not. > > > > Looking forward to your further

Experiences testing new producer performance across multiple threads/producer counts

2015-05-13 Thread Garry Turkington
Hi, I talked with Gwen at Strata last week and promised to share some of my experiences benchmarking an app reliant on the new producer. I'm using relatively meaty boxes running my producer code (24 core/64GB RAM) but I wasn't pushing them until I got them on the same 10GB fabric as the Kafka

RE: Building a producer/consumer supporting exactly-once messaging

2014-02-10 Thread Garry Turkington
a lot of the more fine grained control of offsets (e.g. to reset your position). We will be working on that soon. -Jay On Mon, Feb 10, 2014 at 9:11 AM, Garry Turkington < g.turking...@improvedigital.com> wrote: > Hi, > > I've been doing some prototyping on Kafka for a fe

Building a producer/consumer supporting exactly-once messaging

2014-02-10 Thread Garry Turkington
Hi, I've been doing some prototyping on Kafka for a few months now and like what I see. It's a good fit for some of my use cases in the areas of data distribution but also for processing - liking a lot of what I see in Samza. I'm now working through some of the operational issues and have a que