Hi guys,
As we know, the new consumer of 0.9 kafka does not need to communicate with
zookeeper.
What is the motive / rational behind this disign? I have some
guesses(correct me if i am wrong):
1. Reduce the complexity of consumer. It's hard to maintain the connections
and states of kafka servers
Hi, i am wondering why the increased replication factor is related to a
partition instead of the topic. Isn't it very hard to manage? Any one can
help me clarify this?
On Nov 27, 2015 6:51 AM, "Dillian Murphey" wrote:
> Alright, thank you all. Appreciate it.
>
> Cheers
>
> On Wed, Nov 25, 2015 at
Your producer produces messages faster than the broker can consume it.
You can
1. set buffer size for producer (queue.buffering.max.messages) larger to
reduce the frequency of sending message
2. Add more brokers/partitions to consume the message.
On Mon, Nov 2, 2015 at 11:33 PM, Eduardo Alfaia
wr
Hi, according to my undersanding, your scenario does not apply here. Async
does not mean it buffers message when connection is lost(you killed the
broker). If the connection is down, the producer should detect it as a
exceptional condition, and throw this exception to application level to
handle it
For producer, there is no need to know zookeeper servers to produce
messages.
For consumer, it was necessary to connect zookeeper to consume messages,
because previously the offsets value were
stored by zookeeper, consumer need to know the offsets value to read
message from kafka brokers. Newest v
Hi, though i don't fully understand your question, i'd like to comment on
your code design.
1. it's better for you to run you consumer in a separate thread instead of
main thread, because it blocks the execution main thread.
2. you created a thread for each message, this is very costly. if it take
I am interested in this question as well. Can someone share insights on
this topic?
On Thu, Oct 15, 2015 at 9:34 PM, Hendrik Dev wrote:
> Does Kafka rely on synchronized server clocks?
>
> Or with other words: Will Kafka/Zookeeper work reliable when server
> clocks are not synchronized (with tha
It will never happan.
On Thu, Oct 1, 2015 at 4:22 AM, Chad Lung wrote:
> I seen a previous question (http://search-hadoop.com/m/uyzND1lrGUW1PgKGG)
> on offset rollovers but it doesn't look like it was ever answered.
>
> Does anyone one know what happens when an offset max limit is reached?
> Ove
Hi there,
I noticed that there are several producers our there:
** **org.apache.kafka.clients.producer.KafkaProducer*
** **kafka.javaapi.producer.Producer*
What's the difference between them? Which one should be used?
Hi, kafka is a partitioned system, so not all the servers have the complete
data set.
The producer does not know whether the topic is not exist, or some brokers
are down, it just tries to query again and again until timeout occurs. I
think org.apache.kafka.common.errors.TimeoutException makes sens
As a cluster, each none in the cluster should know each other to function
properly. For a Cassandra cluster(I don't know ActiveMQ's mechanism), as an
example, it has its own
protocal to communicate with each other to know their condition.
For Kafka, each node stays independantly, they use zookeepe
Hi, I think there is no difference between shutting down a consumer or
killing a consumer.
For the whole system, it only means a consumer has left for some reason
which is not interested.
So if you kill a consumer, some consumer in the same consumer group should
take over and consume messages.
Cor
Kafka.
>
> Best,
>
> Roger
>
> Sent from my iPhone
>
> > On Sep 12, 2015, at 10:34 PM, Li Tao wrote:
> >
> > Hi Hackers,
> >
> > This is Lee, a learner of kafka, i have read the original paper on kafka,
> > and walked through the document.
Hi Hackers,
This is Lee, a learner of kafka, i have read the original paper on kafka,
and walked through the document.
I think the best way to learn sth is to write and read code about it. I am
wondering is there any open source code / system which is based on kafka so
that i can read or contribu
14 matches
Mail list logo
