Thanks Jun, I see your point. I will update this thread if I find anything
interesting.
On Tue, Mar 4, 2014 at 8:30 PM, Jun Rao wrote:
> ZK observers only help the latency on the reads, but not writes. Kafka
> consumer need to write to ZK for owning partitions and committing offsets.
>
> Consum
Currently, message offsets are not preserved by mirror maker.
You can potentially do the failover based on the failover time. Suppose
that the consumption in A failed at time t. You find the offset before time
t using our getOffsetBefore api to get the starting offset in B. Then, you
have to manua
Yes, the produce request itself won't trigger topic creation. However, the
first thing the producer client does is to issue a metadata request.
Thanks,
Jun
On Tue, Mar 4, 2014 at 5:49 PM, Guozhang Wang wrote:
> Hi Churu,
>
> I think only 3) will trigger the topic creation, we need to change t
ZK observers only help the latency on the reads, but not writes. Kafka
consumer need to write to ZK for owning partitions and committing offsets.
Consuming from a remote DC is actually fine though. Kafka consumer only
uses ZK during rebalances, which should be rare. You probably want to set a
larg
Just change config/log4j/properties inside Kafka.
Thanks,
Jun
On Tue, Mar 4, 2014 at 4:09 PM, 김동경 wrote:
> I couldn`t find any configuration relevant to turning off the log in
> http://kafka.apache.org/documentation.html#configuration.
> I included Kafka as Maven dependency.
> How could I tur
Thanks for the reply Guozhang. See inline comments:
On Tue, Mar 4, 2014 at 5:54 PM, Guozhang Wang wrote:
>
> 1) is hard to do since ZK stores both broker and consumer metadata, and
> consumers use ZK to also discover brokers.
>
Thanks for the confirmation.
Not sure if I understand 2),
The co
Hi Arup,
1) is hard to do since ZK stores both broker and consumer metadata, and
consumers use ZK to also discover brokers.
Not sure if I understand 2), 3) is essentially equivalent to having a
remote consumer since the KafakMirror actually have a consumer consuming
from the source (mostly remote
Hey guys,
It would be good to tag any JIRA for something which is an confusing or
annoying with the "usability" tag. I am trying to get a list of all these
together so we can take a wack at some of them in a co-ordinated way.
-Jay
Hi Churu,
I think only 3) will trigger the topic creation, we need to change the
misleading description. Thanks for pointing out.
Guozhang
On Tue, Mar 4, 2014 at 3:10 PM, Churu Tang wrote:
> Hi,
>
> According to the auto.create.topics.enable description, 1). attempts to
> produce, 2), consume
Hi,
According to the auto.create.topics.enable description, 1). attempts to
produce, 2), consume, 3), fetch metadata for a non-existent topic will
automatically create the topic with default config. I set the
auto.create.topics.enable to true, and only 3) fetch metadata will creates the
topic.
Hi,
According to the auto.create.topics.enable description, 1). attempts to
produce, 2), consume, 3), fetch metadata for a non-existent topic will
automatically create the topic with default config. I set the
auto.create.topics.enable to true, and only 3) fetch metadata will creates the
topic.
I couldn`t find any configuration relevant to turning off the log in
http://kafka.apache.org/documentation.html#configuration.
I included Kafka as Maven dependency.
How could I turn off the Kafka log in the code?
Thanks
Regards
Dongkyoung
2014-03-04 14:40 GMT+09:00 Jun Rao :
> I think it tries t
Hi,
We have a use case where we have the kafka cluster in a different
datacenter from the consumers. The kafka cluster is in the same data center
where producers are. I have heard that zookeeper could be slow if the
zookeper clients are in different geo location. Is there any best practice
for thi
Hi,
I have a question about mirroring. I would like to create a highly
available Kafka service that runs on AWS and can survive an AZ failure.
Based on what I've read, I plan to create a Kafka cluster in each AZ and
use mirror maker to replicate one cluster to the other. I'll call the two
clus
On Tue, Mar 4, 2014 at 10:33 AM, Guozhang Wang wrote:
> Hi David,
>
> When you (re)-start the producer/consumer, the broker list is used for
> bootstrap, so it should guarantee that some of the brokers listed is alive
> when the client starts. When you migrate from 1,2,3 to 4,5,6 (in fact, in
> t
Hi David,
When you (re)-start the producer/consumer, the broker list is used for
bootstrap, so it should guarantee that some of the brokers listed is alive
when the client starts. When you migrate from 1,2,3 to 4,5,6 (in fact, in
this case it is better to just keep the broker id but just replace t
16 matches
Mail list logo
