Mirror Maker 2 with different Avro Schema Registries

Hi! I'm trying to replicate a few topics using Mirror Maker 2 (2.8). Both the source and the target cluster use Schema Registry (Karapace, on Aiven) and the Confluent Avro serialization format when publishing messages. This means that messages replicated from source->target are not readable with

Re: [VOTE] 2.8.1 RC0

Hi Randall, Thanks for your comments. I definitely missed updating https://github.com/apache/kafka/blob/2.8.1-rc0/docs/js/templateData.js. I will fix this. I will cut a new RC to address this. Regarding https://github.com/apache/kafka-site/tree/asf-site/28, I did not update the documentation the

[VOTE] 2.8.1 RC1

Hello Kafka users, developers and client-developers, This is the second candidate for release of Apache Kafka 2.8.1. Apache Kafka 2.8.1 is a bugfix release and fixes 49 issues since the 2.8.0 release. Please see the release notes for more information. Release notes for the 2.8.1 release: https:/

Re: [VOTE] 2.8.1 RC0

I have started a new thread to vote for the RC1. Therefore, this vote is close. Thanks, David On Tue, Sep 14, 2021 at 11:14 AM David Jacot wrote: > Hi Randall, > > Thanks for your comments. > > I definitely missed updating > https://github.com/apache/kafka/blob/2.8.1-rc0/docs/js/templateData.js

Re: Mirror Maker 2 with different Avro Schema Registries

Hey Anders, take a look at Connect's serdes and SMTs. MirrorMaker can be configured to use them. Ryanne On Tue, Sep 14, 2021, 3:13 AM Anders Engström wrote: > Hi! > I'm trying to replicate a few topics using Mirror Maker 2 (2.8). > > Both the source and the target cluster use Schema Registry (K

Open-source real workload traces for benchmarking distributed streaming systems.

Hi all, Can you please share any open-source real workload traces (events arrivals per unit time and their interarrival distribution) from different sources/businesses to benchmark the performance streaming platforms like Kafka streams? e.g., a credit card payment transactions traces (say for fr

Re: [VOTE] 2.8.1 RC0

David, I am working on a solution for this that would allow us to validate the site docs locally with Docker containers without having to point to a LIVE location I will share an update soon On Tue, Sep 14, 2021 at 5:15 AM David Jacot wrote: > Hi Randall, > > Thanks for your comments. > > I d

Re: Mirror Maker 2 with different Avro Schema Registries

Thanks Ryanne! I'll look into that. I was hoping for an "easier" solution, not having to write custom code for this. But that's probably where we'll end up. Does anyone know if mirror-makers hard-coded ByteArraySerialization for consumers/producers is by design? /Anders On Tue, Sep 14, 2021 at 3:5

Re: [VOTE] 2.8.1 RC1

Thanks for generating a new RC1 with the corrected site docs, David. I was able to successfully complete the following: - Installed 2.8.1 RC0 and performed quickstart for broker and Connect - Verified signatures and checksums - Verified the tag - Manually compared the release notes to JIRA - Buil

Re: [VOTE] 2.8.1 RC1

Hi, +1 (binding) - verified the signatures - ran the tests on the source archive - verified the quickstart with binary - verified the artifacts, javadoc Thanks for running the release! Thanks, Manikumar On Tue, Sep 14, 2021 at 10:36 PM Chris Egerton wrote: > Hi David, > > I took a look at th

Re: [VOTE] 2.8.1 RC1

Hi David, I took a look at the Javadocs and noticed a small issue. Using the search bar at the top of the landing page ( https://home.apache.org/~dajac/kafka-2.8.1-rc1/javadoc/), I entered "KafkaProducer" and clicked on the search item that came up. This brought me to https://home.apache.org/~daja

Re: [VOTE] 2.8.1 RC1

Hi Chris and David, Regarding the API search, this is a known bug in JDK 11 and prior. It can't correctly generate URLs for projects not using the module system (Kafka) that link to projects that use it (Java Standard Library). The bug has been fixed in JDK 12 and higher. To have the search work a

Re: Mirror Maker 2 with different Avro Schema Registries

Yes it's a consequence of Connect, which has its own serde model. mirror-makers hard-coded ByteArraySerialization for > consumers/producers

Re: [VOTE] 2.8.1 RC1

Josep, This is fixed in 3.0.0, right? Ismael On Tue, Sep 14, 2021 at 11:32 AM Josep Prat wrote: > Hi Chris and David, > > Regarding the API search, this is a known bug in JDK 11 and prior. It can't > correctly generate URLs for projects not using the module system (Kafka) > that link to projec

Re: [VOTE] 2.8.1 RC1

Hi Ismael, Yes the 'fix' was merged in the 3.0.0 branch (trunk at that time). I also checked the current 3.0.0 RC2 and the javadoc search works as expected. Best, ——— Josep Prat Aiven Deutschland GmbH Immanuelkirchstraße 26, 10405 Berlin Amtsgericht Charlottenburg, HRB 209739 B Geschäftsführ

Re: [VOTE] 3.0.0 RC2

Hi Konstantine, Thanks for running the release I ran the following checks: - PGP Signatures used to sign the release artifacts - Validation of Release Artifacts Cryptographic Hashes (ASC MD5 SHA1 SHA512) - Validation of Kafka Source and Tests - Validation of Kafka Site Documentati

Re: [kafka-clients] Re: [VOTE] 2.8.1 RC1

Hi Community Members, To encourage other community members to participate in the release candidate validations and voting, I have set up the following resource as part of the work for KAFKA-9861 It is a set of scripts and Docker images that allows community members to run local validations in a c