hi everyone,
we are trying to deploy Kafka 0.8.2.1 and Zookeeper on AWS using
Cloudformation, ASG's and other Services. For Zookeeper we are using
Netflix' Exhibitor (V 1.5.5) to ensure failover stability.
What we are observing right now is that after some days our Brokers are not
registered anym
Hello Apache Kafka community,
If I recall well, two weeks ago it was mentioned in a discussion that Kafka
0.8.3 might be released in a month time.
Is this still Kafka dev team goal, in few weeks time to have Kafka 0.8.3
released? Or is more (re)work (e.g. more new consumer API changes) planned
fo
Hello guys,
A while ago i read that the new consumer api was going to be released
sometime in July as part of the 0.8.3/0.9 release.
https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan
Do we have an update when we think that can happen?
Thanks,
Jalpesh
-- PRIVILEGED AND C
Hello,
Is there a way that we can find the lead broker
from kafka.message.MessageAndMetadata class?
My use case is simple, I have topic and partition and wanted to find out
the lead broker for that partition.
Please provide your insights
Thanks
Sreeni
Hi,
I need to know how many consumers can subscribe to a single topic ( with
one partition ).
I have a requirement such that,
1. The producer will post to 1 topic having 1 partition.
1. *20,000-30,000 consumers *should to be able to consume messages.
2. All of the consumers should rec
Hi, Stevo,
Yes, we are still iterating on the new consumer a bit and are waiting for
some of the security jiras to be committed. So now, we are shooting for
releasing 0.8.3 in Oct (just updated
https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan).
As we are getting closer, we wi
Jalpesh,
We are still iterating on the new consumer a bit and are waiting for some
of the security jiras to be committed. So now, we are shooting for
releasing 0.8.3 in Oct (just updated
https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan).
Thanks,
Jun
On Mon, Aug 3, 2015 at 8
Hey Valibhav,
With only one partition, all of the consumers will end up hitting a single
broker (since partitions cannot be split). Whether it is possible to get
that number of consumers on a single broker may depend on the message load
through the topic. I think there has been some interest in al
I don't know a specific limit for number of consumers, perhaps someone will
have better idea.
Do note that with a single topic and a single partition, you can't really
scale by adding more machines - the way Kafka is currently designed, all
consumers will read from the one machine that has the lea
I don't know of any limits as such but I don't think your problem is suitable
for Kafka. The third point especially wouldn't work with Kafka. Using Kafka,
only one consumer will get a message out of the 30k
> On 03-Aug-2015, at 10:39 am, Vaibhav Kirte wrote:
>
> Hi,
>
> I need to know how m
https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan
Any idea when 0.9.0 will be released?
Thanks,
Shri
This message and its contents (to include attachments) are the property of
National Health Systems, Inc. and may contain confidential and pr
According to the plan, never :)
Is there a specific feature you are looking forward to? I think the most
exciting features are planned for 0.8.3 - which is targeted for Oct.
On Mon, Aug 3, 2015 at 1:14 PM, Shrikant Patel wrote:
> https://cwiki.apache.org/confluence/display/KAFKA/Future+release+
According to
https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetCommitRequest,
we can store custom metadata with our checkpoints. It looks like the high
level consumer does not support committing offsets with metadata, and that in
orde
I'm looking at https://issues.apache.org/jira/browse/KAFKA-1684. Will this be
part of 0.8.3??
We are in healthcare domain, its company security requirement to have anything
on wire to be encrypt. So producers to broker and broker to consumer needs to
be encrypted.
I think the answer is yes, I was confused because on Jira it say 0.9.0.
__
Shrikant Patel | 817.246.6760 | ext. 4302
Enterprise Architecture Team
PDX-NHIN-Rx.com
-Original Message-
From: Shrikant Patel [mailto:spa...@pdxinc.com]
Sent: Mo
Yep. Fixed the JIRA.
Thanks :)
On Mon, Aug 3, 2015 at 2:01 PM, Shrikant Patel wrote:
> I think the answer is yes, I was confused because on Jira it say 0.9.0.
>
> __
> Shrikant Patel | 817.246.6760 | ext. 4302
> Enterprise Architecture Team
>
You are correct. You can see that ZookeeperConsumerConnector is hardcoded
with null metadata.
https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala#L310
More interesting, it looks like the Metadata is not exposed in the new
KafkaConsumer eit
Nice new email address, Gwen. :)
On Aug 3, 2015, at 3:17 PM, Gwen Shapira wrote:
> You are correct. You can see that ZookeeperConsumerConnector is hardcoded
> with null metadata.
> https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala#L310
If it was my architecture, I'd consider publishing a "control message"
either to same topic or separate saying "done with topic A".
The "thing" that needs to continue copying A also needs to know where to
continue, so presumably if the "done with A" message is missing, it will
read from sorted-topi
Hi I have a very simple code to just send out one message, the topic is
created automatically, but the message just cannot be sent out. I also
tried to change the configuration, the result is still the same. Sorry to
bother you all with this silly question.
For your information, the kafka server i
20 matches
Mail list logo
