Kafka topic registration documentation.

2015-06-09 Thread Piotr Husiatyński
Hi, I was wondering if the process of new topic registration is described somewhere. I'm using Go client to talk to kafka cluster and it would be awesome to ensure topic existence. I know there is auto creation configuration flag, but using this there is no way to use custom configuration for ever

Re: Fetch API Offset

2015-04-22 Thread Piotr Husiatyński
t; > On Tue, Apr 21, 2015 at 6:46 PM, Piotr Husiatyński wrote: > >> I'm using Go with kafka library** and sending fetch request as >> described in wire protocol documentation returns messages off by one. >> Are you sure that code that you wrote is not sending fetch re

Re: Fetch API Offset

2015-04-21 Thread Piotr Husiatyński
> The only caveat is that fetch returns a block of messages which means I > might > get messages with lower offset that I just need to ignore. > > regards > > > On Tue, Apr 21, 2015 at 6:47 AM, Piotr Husiatyński wrote: > >> What do you mean that message has expired? &

Re: Fetch API Offset

2015-04-21 Thread Piotr Husiatyński
wrote: > Hi, > > I suspect If some message from the given offset have expired, then they > will not be returned. > > regards > > On Tue, Apr 21, 2015 at 5:14 AM, Piotr Husiatyński wrote: > >> According to documentation, sending fetch request with offset value &g

Fetch API Offset

2015-04-21 Thread Piotr Husiatyński
According to documentation, sending fetch request with offset value result in messages starting with given offset (including) and greater. https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-FetchAPI > Logically one specifies the topics, partit

Re: New kafka client for Go (golang)

2015-04-07 Thread Piotr Husiatyński
Sorry if this mail is not send properly, but I have no idea how to send in gmail response to mailing list without having original message in mailbox. > How does it compare to Sarama? There are several important differences. It's been about two months since I looked at sarama API and I know that

New kafka client for Go (golang)

2015-03-30 Thread Piotr Husiatyński
Hi, I wanted to share new client library for Go language that I'm developing at Optiopay. Library is MIT licensed and provides API close to latest kafka request/response wire format with few higher level helpers to make common tasks easier. The only thing that is not yet implemented is message set

Producing message set

2015-02-11 Thread Piotr Husiatyński
Hi, I'm writing new client library for kafka and I cannot find information about atomicity of message set publishing. I'm implementing protocol described on wiki [1]. According to documentation, it is possible to send within single request more than one message - message set for any partition. In