Re: Consumer of multiple topic

2015-10-23 Thread Fajar Maulana Firdaus
oh that's great. thank you On Fri, Oct 23, 2015 at 10:39 PM, pushkar priyadarshi wrote: > Currently there are no partitions based subscription inside topic.So when > you subscribe to both topics your consumer will get data from each > partitions in these two topics, i dont think you would be miss

Kafka Mirror Maker & Encryption

2015-10-23 Thread Tim Krieger
Hey Guys, I have been hunting through the archives and can't find anyone else that has ran into this issue so I'm going to toss it out there an hope someone can point me to a solution. We are running two kafka clusters on opposite coasts in AWS and need to have mirror maker copy the messages from

drop in fetch-follower requests

2015-10-23 Thread Wesley Chow
We deployed a Samza process that effectively doubled our fetch rates in our cluster and noticed a significant drop fetch-follower requests. What is the difference between fetch-consumer and fetch-follower? [image: Inline image 1] We noticed that some topics moved along undisturbed, sending in the

Re: How can i secure Kafka topics

2015-10-23 Thread Parth Brahmbhatt
If you are ok with SSL instead of kerberos follow this guide https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka The patch to support for kerberos just went in see https://issues.apache.org/jira/browse/KAFKA-1686. But the usage doc still needs to be created. On 10/23/15, 10:

How can i secure Kafka topics

2015-10-23 Thread prateek arora
Hi Can i secure Kafka topics like Apache hbase and hdfs have kerberos security mechanism ? i am not able to find out any document related to Kafka security . please suggest how can i secure Kafka . Regards Prateek

Re: Questions about .9 consumer API

2015-10-23 Thread Mohit Anchlia
Can I get a link to other type of examples? I would like to see how to write the API code correctly. On Fri, Oct 23, 2015 at 8:23 AM, Gwen Shapira wrote: > There are some examples that include error handling. These are to > demonstrate the new and awesome seek() method. > You don't have to handl

Re: future of Camus?

2015-10-23 Thread Adrian Woodhead
Thanks everyone for your input on this thread, looks like a hot topic ;) I thought I'd reply to everyone's feedback in one go rather than have lots of separate replies, so here goes... Henry - thanks for pointing out Secor, I had never seen it before. I can see why not having a Hadoop dependenc

Re: Consumer of multiple topic

2015-10-23 Thread pushkar priyadarshi
Currently there are no partitions based subscription inside topic.So when you subscribe to both topics your consumer will get data from each partitions in these two topics, i dont think you would be missing anything. On Fri, Oct 23, 2015 at 11:35 AM, Fajar Maulana Firdaus wrote: > I am using k

Consumer of multiple topic

2015-10-23 Thread Fajar Maulana Firdaus
I am using kafka to implement event sourcing. Based on a threat in http://search-hadoop.com/kafka, I created sepparate topic for my events, e.g. UserCreatedEvent and UserUpdateEmailEvent. Both events are using userId as the message key. However assuming each topic has several partition, how to setu

Re: Questions about .9 consumer API

2015-10-23 Thread Gwen Shapira
There are some examples that include error handling. These are to demonstrate the new and awesome seek() method. You don't have to handle errors that way, we are just showing that you can. On Thu, Oct 22, 2015 at 8:34 PM, Mohit Anchlia wrote: > It's in this link. Most of the examples have some ki