Kafka upgraded from 3.5.1 to 3.7.0 version On Fri, May 10, 2024 at 2:13 AM Sophie Blee-Goldman <sop...@responsive.dev> wrote:
> What version did you upgrade from? > > On Wed, May 8, 2024 at 10:32 PM Penumarthi Durga Prasad Chowdary < > prasad.penumar...@gmail.com> wrote: > > > Hi Team, > > I'm utilizing Kafka Streams to handle data from Kafka topics, running > > multiple instances with the same application ID. This enables distributed > > processing of Kafka data across these instances. Furthermore, I've > > implemented state stores with time windows and session windows. To > retrieve > > windows efficiently, I've established a remote query mechanism between > > Kafka Streams instances. By leveraging the queryMetadataForKey method on > > streams, I can retrieve the hostname where a specific key was processed > and > > where the corresponding window data resides in the state store. > > *streams.queryMetadataForKey(storeName, recordKey, new > > DataKeySerilizer()).activeHost();* > > This functionality has been running smoothly for quite some time, up > until > > we upgraded our Kafka and Kafka Streams versions to 3.7.0. Since the > > upgrade, we've noticed some unexpected behavior that we didn't encounter > > with the previous versions. > > > > - The queryMetadataForKey method is returning "unavailable" as the > > hostname, despite having two Kafka Streams instances in a running > state. > > The issue seems to persist intermittently, and restarting the Kafka > > Streams > > instances temporarily resolves it. However, the problem resurfaces > after > > some time. > > - Additionally, we've observed that the state store of Kafka Streams > > instances is not evenly distributed as it was before the upgrade. > > Previously, if a Kafka topic had 10 partitions and two Kafka Streams > > instances were running, these 10 partitions would be evenly shared > > between > > the two instances. However, this behavior seems to have changed after > > the > > upgrade. > > > > When can this issue happen? > > How can I fix the issue ? > > I would like to express my gratitude in advance for any assistance > > provided. > > -- > > > > > > > > Thank's&Regard's, > > Prasad, > > 91-9030546248. > > > -- Thank's&Regard's, Prasad, 91-9030546248.
