Re: Apache Kafka website "videos" page clarification

2020-09-10 Thread Ben Stopford
Hi Paulo The reason for using Kafka Summit videos is there is an extensive, community-driven selection process that goes into Kafka Summit driven by the Kafka Summit Program Committee. This is then further filtered by the community itself: those members of the community that attend the summit and

Re: Error in kafka streams: The producer attempted to use a producer id which is not currently assigned to its transactional id

2020-09-10 Thread Matthias J. Sax
If KafkaStreams goes into ERROR state, you cannot do anything but restart it. We constantly improve KafkaStreams to avoid getting into ERROR state but it's not always possible to auto-recover. The point being: You should try to figure out the root cause for the issue (not easy to do unfortunately

Questions around Time based retention policy

2020-09-10 Thread nitin agarwal
Hi All, I have a few questions around time based retention policy. It will be really helpful if you can answer. 1. Does Kafka use CreateTime for building its time based indices? Does CreateTime have any role in deciding time based log rotation ? 2. If CreateTime is provided by the clients then wou

Go Library that uses writer and reader schema

2020-09-10 Thread Scott Reynolds
Hi List, TL;DR Looking for a Go library that can Decode binary data written with a different Schema then the Reader. We have a system in place that allows for multiple Avro schemas to be live for data streams. There can be producers producing an earlier version of the schema (version 1) onto the

MirrorMaker 2 - Can we start mirroring from latest?

2020-09-10 Thread Samuel Cantero
Hello, I'd like to know whether starting mirroring from latest is doable with mirrormaker2. I tried setting the target-cluster-alias.consumer.auto.offset.reset to latest but it doesn't work (for an active/passive setup). Any help would be appreciated. Best,

Re: Request for Comments/Resolution - KAFKA-10465

2020-09-10 Thread Guozhang Wang
Hello, Thanks for pinging the community. I've taken a look at your code at KAFKA-10465 and replied in the ticket. Guozhang On Mon, Sep 7, 2020 at 8:46 AM M. Manna wrote: > Hello, > > We do appreciate that release 2.7 is keeping us occupied, but this bug (or > not) is holding us back from makin