Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

Thanks for the input Ashwin. > 1. Can you elaborate on the rejected alternatives ? Suppose connector > config is versioned and has a schema. Then a single plugin (whose > dependencies have not changed) can handle multiple config versions for the > same connector class. IIUC (please correct me if

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1372

See

Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

Hi Snehasis, This is a really useful feature and thanks for initiating this discussion. I had the following questions - 1. Can you elaborate on the rejected alternatives ? Suppose connector config is versioned and has a schema. Then a single plugin (whose dependencies have not changed) can hand

Jenkins build is still unstable: Kafka » Kafka Branch Builder » trunk #1371

See

Re: [DISCUSS] KIP-890 Server Side Defense

Thanks for the KIP. Couple of clarification questions (I am not a broker expert do maybe some question are obvious for others, but not for me with my lack of broker knowledge). (10) The delayed message case can also violate EOS if the delayed message comes in after the next addPartitions

Re: [DISCUSS] KIP-889 Versioned State Stores

Thanks for the KIP Victoria. Very well written! Couple of questions (many might just require to add some more details to the KIP): (1) Why does the new store not extend KeyValueStore, but StateStore? In the end, it's a KeyValueStore? (2) Should we have a ReadOnlyVersionedKeyValueStore? E

Re: [DISCUSS] KIP-864: Add End-To-End Latency Metrics to Connectors

Hi Jorge, Thanks for the updates, and apologies for the delay. The new diagram directly under the "Proposed Changes" section is absolutely gorgeous! Follow-ups: RE 2: Good point. We can use the same level for these metrics, it's not a big deal. RE 3: As long as all the per-record metrics are k

Re: [VOTE] KIP-884: Add config to configure KafkaClientSupplier in Kafka Streams

+1 (binding) On 11/21/22 7:39 AM, John Roesler wrote: I'm +1 (binding) Thanks for the KIP! -John On 2022/11/17 21:06:29 Hao Li wrote: Hi all, I would like start a vote on KIP-884: https://cwiki.apache.org/confluence/display/KAFKA/KIP-884%3A+Add+config+to+configure+KafkaClientSupplier+in+Kaf

Jenkins build is unstable: Kafka » Kafka Branch Builder » trunk #1370

See

Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

Hi Mickael. Thanks for your input. Addressing the point you mentioned below. > 1) Can you explain how this would work with the GET > /{pluginName}/config endpoint? How do you specify a version for a > connector? This API returns the set of configurations for a given connector. Since between versi

RE: Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

Hi Mickael. Thanks for your input. Addressing the point you mentioned below. > 1) Can you explain how this would work with the GET > /{pluginName}/config endpoint? How do you specify a version for a > connector? This API returns the set of configurations for a given connector. Since between versi

[jira] [Created] (KAFKA-14414) Remove unnecessary usage of ObjectSerializationCache

Divij Vaidya created KAFKA-14414: Summary: Remove unnecessary usage of ObjectSerializationCache Key: KAFKA-14414 URL: https://issues.apache.org/jira/browse/KAFKA-14414 Project: Kafka Issue Ty

[jira] [Created] (KAFKA-14413) Separate MirrorMaker configurations for each connector

Mickael Maison created KAFKA-14413: -- Summary: Separate MirrorMaker configurations for each connector Key: KAFKA-14413 URL: https://issues.apache.org/jira/browse/KAFKA-14413 Project: Kafka Is

Re: [VOTE] KIP-884: Add config to configure KafkaClientSupplier in Kafka Streams

I'm +1 (binding) Thanks for the KIP! -John On 2022/11/17 21:06:29 Hao Li wrote: > Hi all, > > I would like start a vote on KIP-884: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-884%3A+Add+config+to+configure+KafkaClientSupplier+in+Kafka+Streams > > > Thanks, > Hao >

Re: [DISCUSS] KIP-852 Optimize calculation of size for log in remote tier

Thank you Jun and Alex for your comments. Point#1: You are right Jun. As Alex mentioned, the "derived metadata" can increase the size of cached metadata by a factor of 10 but it should be ok to cache just the actual metadata. My point about size being a limitation for using cache is not valid anym

Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

Hi, Thanks for the KIP, this is something that could be really useful! 1) Can you explain how this would work with the GET /{pluginName}/config endpoint? How do you specify a version for a connector? 2) Some connectors come bundled with transformations (for example Debezium). How would multiple

[DISCUSS] KIP-892: Transactional Semantics for StateStores

Hi everyone, As I mentioned in the discussion thread for KIP-844, I've been working on an alternative approach to achieving better transactional semantics for Kafka Streams StateStores. I've published this separately as KIP-892: Transactional Semantics for StateStores

[jira] [Created] (KAFKA-14412) Transactional semantics for StateStores

Nicholas Telford created KAFKA-14412: Summary: Transactional semantics for StateStores Key: KAFKA-14412 URL: https://issues.apache.org/jira/browse/KAFKA-14412 Project: Kafka Issue Type: I

Build failed in Jenkins: Kafka » Kafka Branch Builder » trunk #1369

See Changes: -- [...truncated 512952 lines...] [2022-11-21T14:47:33.438Z] Gradle Test Run :streams:integrationTest > Gradle Test Executor 168 > IQv2IntegrationTest > shoul

Re: Request for Kafka Jira account

Hi, I've created your account, you should receive an email with all the details. Thanks, Mickael On Mon, Nov 21, 2022 at 3:23 PM Gantigmaa Selenge wrote: > > Hi team, > > Can I please have an account created for Kafka Jira? > > Username: tinaselenge > Display name: Gantigmaa Selenge > Email add

Request for Kafka Jira account

Hi team, Can I please have an account created for Kafka Jira? Username: tinaselenge Display name: Gantigmaa Selenge Email address: tina.sele...@gmail.com Thanks! Regards, Gantigmaa Posted to dev@kafka.apache.org

Re: [DISCUSS] KIP-844: Transactional State Stores

Hi Alex, Thanks for getting back to me. I actually have most of a working implementation already. I'm going to write it up as a new KIP, so that it can be reviewed independently of KIP-844. Hopefully, working together we can have it ready sooner. I'll keep you posted on my progress. Regards, Ni

[jira] [Created] (KAFKA-14411) Logging warning when partitions don't exist on assign request

Paolo Patierno created KAFKA-14411: -- Summary: Logging warning when partitions don't exist on assign request Key: KAFKA-14411 URL: https://issues.apache.org/jira/browse/KAFKA-14411 Project: Kafka

Re: [DISCUSS] KIP-844: Transactional State Stores

Hey Nick, Thank you for the prototype testing and benchmarking, and sorry for the late reply! I agree that it is worth revisiting the WriteBatchWithIndex approach. I will implement a fork of the current prototype that uses that mechanism to ensure transactionality and let you know when it is read

Re: [DISCUSS] KIP-875: First-class offsets support in Kafka Connect

Hi Chris, Thanks for all the updates, yes that seems good! Mickael On Thu, Nov 17, 2022 at 8:41 PM Chris Egerton wrote: > > Hi Mickael, > > Thanks for your thoughts! IMO it's most intuitive to use a null value in > the PATCH API to signify that an offset should be reset, since it aligns > nicel

Build failed in Jenkins: Kafka » Kafka Branch Builder » 3.3 #123

See Changes: -- [...truncated 500325 lines...] [2022-11-21T05:29:38.183Z] org.apache.kafka.streams.integration.StreamStreamJoinIntegrationTest > testOuterRepartitioned[cachi

[DISCUSS] KIP-891: Running multiple versions of a connector.

Hi all, I'd like to start a discussion thread on KIP-891: Running multiple versions of a connector. The KIP aims to add the ability for the connect runtime to run multiple versions of a connector. https://cwiki.apache.org/confluence/display/KAFKA/KIP-891%3A+Running+multiple+versions+of+a+connect

[jira] [Created] (KAFKA-14410) Allow connect runtime to run multiple versions of a connector.

Snehashis Pal created KAFKA-14410: - Summary: Allow connect runtime to run multiple versions of a connector. Key: KAFKA-14410 URL: https://issues.apache.org/jira/browse/KAFKA-14410 Project: Kafka