dear Kafka users,
Is there any tips about how to configure *offsets.load.buffer.size*
configuration to speed up the offsets loading after leader change? The
default value for this configuration is 5242880
Thank you,
--
Muqtafi Akhmad
Software Engineer
Traveloka
Hi all;
I try to publish/consume my java objects to kafka. I use Avro schema.
My basic program works fine. In my program i use my schema in the producer
(for encoding) and consumer (decoding).
If i publish different objects to different topics( eg: 100 topics)at the
receiver, i do not know, what
Yes, you're right. No need to seek the offsets in onPartitionsAssigned
method internally KafkaConsumer handles it.
Thanks for sharing this with me. I'll update it.
--Kamal
On Mon, Apr 18, 2016 at 7:29 PM, Florian Hussonnois
wrote:
> Yes, but the ConsumerRebalanceListener is optional and by the
Thanks, this seems to do the trick.
Best regards,
Marko
www.kafkatool.com
> Hi
>
> For the Kafka Consumer, there are seekToBeginning and seekToEnd method
> that
> point your the beginning and end of the partition. You can use one of the
> methods to point the consumer to a certain position and u
I figured it out. It seems that Gradle completely ignores
the sourceCompatibility & targetCompatibility properties as it relates to
compiling Scala code.
I was able to get Kafka's Scala code to compile to Java 8 bytecode by
adding the following to the build.gradle:
task
To follow up with my last email, I have been looking into
socket.receive.buffer.byte as well as socket.send.buffer.bytes. Would it help
to increase the buffer for OOM issue?
All help is appreciated!
Thanks!
-nick
From: "McKoy, Nick"
mailto:nicholas.mc...@washpost.com>>
Date: Monday, April
Hey all,
I have a kafka cluster of 5 nodes that’s working really hard. CPU is around 40%
idle daily.
I looked at the file descriptor note on this documentation page
http://docs.confluent.io/1.0/kafka/deployment.html#file-descriptors-and-mmap
and decided to give it a shot on one instance in the
Harsh,
We looked at some of the other client libraries for .NET and opted to use the
Kafka-Rest proxy.
There are challenges with using it, since it is stateful.
You can only run and connect to a single instance at a time.
By that I mean, if you have 3 consumers in a group you still must connec
Kafka Users,
We are running a Kafka cluster (0.8.2-BETA), recently we have noticed an issue
where consumer threads were not assigned to certain partitions ( for instance
if we have 100 partitions per topic than I see partitions 40 to partitions 60
are unclaimed/unassigned by running consumer
Yes, but the ConsumerRebalanceListener is optional and by the default
KafkaConsumer uses a NoOpConsumerRebalanceListener if no one is provided.
I think the seek() is already done internally when a consumer joins or
quits the group. I'm not sure this line is actually needed.
2016-04-18 15:31 GMT+0
Hello All,
I am facing one issue with high level consumer API. ( Multi threaded
programming )
- I created a topic with 4 partitions.
- I started consumer with 1 thread and it was getting messages from all
4 partitions.
- Then I restarted my consumer with 4 threads, now every thread to
When a new consumer joins to the group, it should start to read data
from where the other consumer left.
--Kamal
On Mon, Apr 18, 2016 at 6:58 PM, Florian Hussonnois
wrote:
> Thank you very much, the example is really helpful.
>
> My last question is : Why is it necessay to seek the consumer off
Thank you very much, the example is really helpful.
My last question is : Why is it necessay to seek the consumer offsets into
the onPartitionsAssigned method ?
https://github.com/omkreddy/kafka-examples/blob/master/consumer/src/main/java/kafka/examples/consumer/advanced/AdvancedConsumer.java#L12
13 matches
Mail list logo
