With the ip -> broker id mapping: We've got 3 servers with three ip's and
id's 1, 2, 3. The 3 servers kept the ip's but the broker id's switched
around so something like:
192.168.0.1 -> 1
192.168.0.2 -> 2
192.168.0.3 -> 3
Then they all stopped and came back as:
192.168.0.1 -> 2
192.168.0.2 -> 3
Hi,
I want to use the high level consumer to read all partitions for a topic, and
know when I have reached "the end". I know "the end" might be a little vague,
since items keep showing up, but I'm trying to get as close as possible. I know
that more messages might show up later, but I want to k
Hi,
with Kafka 0.8 it is possible to add new partitions on newly added brokers
and supply a partition assignment to put the new partitions mainly on the
new brokers (4 and 5 are the new brokers):
bin/kafka-add-partitions.sh --topic scale-test-001 \
--partition 14 \
--replica-assignment-list
4:5,4
Hi,
I'm using the Kafka 8.2.1 version(kafka_2.11-0.8.2.1) and the consumer
offset checker hangs indefinitely and does not return any results. I
enabled the debug for tools and below is the debug statements as seen on
the stdout. Any thoughts or inputs on this will be much appreciated.
command used
Jamie,
Once the patch gets in you can configure kafka brokers with ssl configs
and similarly you can configure producer and consumer with ssl both will
negotiate a ssl connection once its established rest of the communication will
happen over ssl. I am not sure about the timeline but we ar
Thank you Harsha for the reply. Will the patch work with 0.8.2.1? So the
general logic is that we write an agent that sends message via https and then
may be a servlet using Kafka produce class to authenticate with kafka server on
ssl port and so on When will the full implementation be avail
Also it would be great to know if you see the same issue when you don't
have different ip -> broker id mapping. Also it would be great if you can
explain " different ip -> broker id mapping" mean as Becket said.
Thanks,
Mayuresh
On Fri, May 8, 2015 at 9:48 AM, Mayuresh Gharat
wrote:
> It shoul
Dan,
Just to make sure I understand it correctly. What do you mean by different
ip -> broker mapping? Do you mean you changed your broker ip? We have
different mechanism in and producer to get the cluster information.
Consumer get all the information from Zookeeper while producer has to talk
to br
Hi Mayuresh,
Yes, the broker is up and accepting connections. Multiple consumers are
consuming off topics on the broker.
Also I am seeing the issue only with this particular version (2.11-0.8.2.1).
It worked fine with the beta that I was using earlier.
On Fri, May 8, 2015 at 12:45 PM, Mayuresh G
It should do a updateMetadataRequest in case it gets NOT_LEADER_FOR
PARTITION. This looks like a bug.
Thanks,
Mayuresh
On Fri, May 8, 2015 at 8:53 AM, Dan wrote:
> Hi,
>
> We've noticed an issue on our staging environment where all 3 of our Kafka
> hosts shutdown and came back with a different
Is X.X.X.X:9092 up and accepting connections?
I am confused aas in why is it not connecting some other broker if
connection to this broker fails. Can you check if the broker is up?
The way it works is the consumer will send a ConsumerMetadataRequest to one
of the brokers and get the offsetmanager
Hi,
I'm using the Kafka 8.2.1 version(kafka_2.11-0.8.2.1) and the consumer
offset checker hangs indefinitely and does not return any results. I
enabled the debug for tools and below is the debug statements as seen on
the stdout. Any thoughts or inputs on this will be much appreciated.
command used
On Thu, May 7, 2015 at 10:01 PM, Rendy Bambang Junior <
rendy.b.jun...@gmail.com> wrote:
> Hi
>
> - Legacy scala api for producer is having keyed message along with topic,
> key, partkey, and message. Meanwhile new api has no partkey. Whats the
> difference between key and partkey?
>
In the new A
Hi,
We've noticed an issue on our staging environment where all 3 of our Kafka
hosts shutdown and came back with a different ip -> broker id mapping. I
know this is not good and we're fixing that separately. But what we noticed
is all the consumers recovered but the producers got stuck with the
fo
14 matches
Mail list logo
