Re: Consistent replication of an event stream into Kafka

2014-05-20 Thread Bob Potter
ence/display/KAFKA/Idempotent+Producer > > Do you think this new feature will solve your case? > > Guozhang > > > On Mon, May 19, 2014 at 2:40 PM, Bob Potter > wrote: > > > Hello, > > > > We have a use case where we want to replicate an event stream

Consistent replication of an event stream into Kafka

2014-05-19 Thread Bob Potter
Hello, We have a use case where we want to replicate an event stream which exists outside of kafka into a kafka topic (single partition). The event stream has sequence ids which always increase by 1. We want to preserve this ordering. The difficulty is that we want to be able to have the process

Re: Truncated messages in fetch response

2013-06-27 Thread Bob Potter
in the next release--the perf hit is pretty > > minor and it is an annoyance and source of bugs for clients. > > > > For now you have to handle it, so I added documentation to the protocol > wiki. > > > > Sorry! > > > > -Jay > > > > On Wed, Jun

Truncated messages in fetch response

2013-06-26 Thread Bob Potter
Howdy, I'm developing a client for kafka 0.8. It looks like a fetch response will sometimes end with a partial message. I understand why this might be the case but it was unexpected and as far as I can tell undocumented. Is my understanding correct or am i missing something? -Bob

Poseidon, a new 0.8 compatible Ruby client for Kafka

2013-02-11 Thread Bob Potter
Howdy, I just pushed the initial version of a new Ruby client that implements the new 0.8 wire protocol and includes a producer which uses the topic metadata API to distribute messages across a cluster. https://github.com/bpot/poseidon It's still very alpha, but I hope to put it through its pace