Re: Consumer poll returning 0 results

2017-10-26 Thread Susheel Kumar
g "--from-beginning" will be equivalent to what you observe above. > > Konstantine > > On Wed, Oct 25, 2017 at 6:48 PM, Ted Yu wrote: > > > Can you provide a bit more information ? > > > > Release of Kafka > > Java / Scala version > > > &

Consumer poll returning 0 results

2017-10-25 Thread Susheel Kumar
Hello Kafka Users, I am trying to run below sample code mentioned in Kafka documentation under Automatic Offset Committing for a topic with 1 partition (tried with 3 and more partition as well). Create command as follows bin/kafka-topics.sh --create --zookeeper :2181 --replication-factor 3 --par

Re: Compact Topic - Handling InProcess/Retry/Manual/Completed status

2016-12-20 Thread Susheel Kumar
our and if they fail, they are moved to *Manual* directory for someone to look manually 6. If successful, Retry files are moved to *Ingested* folder. Thanks, Susheel On Tue, Dec 20, 2016 at 8:27 AM, Susheel Kumar wrote: > Hello, > > I am wondering how we handle the below curren

Compact Topic - Handling InProcess/Retry/Manual/Completed status

2016-12-20 Thread Susheel Kumar
Hello, I am wondering how we handle the below current workflow in Kafka when using compact topic. Do we need to use multiple compact topic's for each of these directories (InProcess/Retry/Manual/Completed) and move messages in between them OR use one topic and application maintains the offset's

Re: Kafka as a database/repository question

2016-12-16 Thread Susheel Kumar
makes you comfortable, maybe take backups > as > > was mentioned. > > > > Hope this helps some > > > > Kenny Gorman > > Founder > > www.eventador.io > > > > > > > On Dec 15, 2016, at 12:00 PM, Susheel Kumar > > wrote: > >

Re: Kafka as a database/repository question

2016-12-15 Thread Susheel Kumar
e enough storage space. Set > replication to something that makes you comfortable, maybe take backups as > was mentioned. > > Hope this helps some > > Kenny Gorman > Founder > www.eventador.io > > > > On Dec 15, 2016, at 12:00 PM, Susheel Kumar > wrote: > >

Re: Kafka as a database/repository question

2016-12-15 Thread Susheel Kumar
riginal Message- > From: Susheel Kumar [mailto:susheel2...@gmail.com] > Sent: Thursday, December 15, 2016 12:00 PM > To: users@kafka.apache.org > Subject: Kafka as a database/repository question > > Hello Folks, > > I am going thru an existing design where Kafka is planned to b

Kafka as a database/repository question

2016-12-15 Thread Susheel Kumar
Hello Folks, I am going thru an existing design where Kafka is planned to be utilised in below manner 1. Messages will pushed to Kafka by producers 2. There will be updates to existing messages on ongoing basis. The expectation is that all the updates are consolidated in Kafka and the