Yes and no :)
1) if you use the index to find the "end-offset" of your scan ("consume
all messages with a smaller offset") you would compare the offset of
each message with this "end offset" -- thus, this is the same thing as
consuming the topic from beginning and just compare the record timestamp
Anyone any thoughts?
If I am not changing this value after upgrade to 1.0 in a single node kafka
broker. It will take the default value of 3 . So what will be the behavior
in this case?
On Fri, Nov 24, 2017 at 3:57 PM, Debraj Manna
wrote:
> Hi
>
> I am migrating from Kafka 0.10 to the latest 1.
I see, thanks. I suppose I was trying to avoid so much custom logic, which is
why I initially was looking at the time-based index functionality. Based on
what you’ve said so far, I take it using the time-based index to find an offset
corresponding to a timestamp and then consuming all messages w
Your understanding is correct.
The simplest way to get more parallelism is to increase the number of
partitions. There is some overhead for this, but it not too much.
>> what you're writing is in sharp contrast with what I know...
I guess, this target other messaging system: Kafka has a differen
Thanks, Faraz.
I am using its Java API. It does not seem to provide such method to the
consumer.
On Wed, Nov 22, 2017 at 2:45 PM, Faraz Mateen wrote:
> Not sure which client you are using.
> In kafka-python, consumer.config returns a dictionary with all consumer
> properties.
>
> Thanks,
> Fara
Hii Sameer,
You can find the current offset of a consumer using method *position *provided
to Kafka Consumer. This method takes the topic partition in parameter to
know which partition you want offset of.
For example, you can do something like
*consumer.position()*
To find the topic partitions as
Hi
I am migrating from Kafka 0.10 to the latest 1.0 . I did not set any value
for these fields in Kafka 0.10. Can some one let me know what is the
recommended settings for a 3 node broker cluster & for a single node broker
cluster for the below internal topic settings -
- offsets.topic.repli
Hi Matthias,
what you're writing is in sharp contrast with what I know...
I read that: "Kafka consumers are typically part of a consumer group. When
multiple consumers are subscribed to a topic and belong to the same
consumer group, each consumer in the group will receive messages from a
differen
Thank you Matthias!
On Thu, Nov 23, 2017 at 10:34 PM, Matthias J. Sax
wrote:
> You might want to consider using the reset tool instead of just changing
> the application.id...
>
> https://www.confluent.io/blog/data-reprocessing-with-kafka-
> streams-resetting-a-streams-application/
>
>
> -Matthi
