[jira] [Resolved] (KAFKA-16326) Kafka Connect unable to find javax dependency on Quarkus update to 3.X

2025-01-30 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-16326. - Resolution: Fixed > Kafka Connect unable to find javax dependency on Quarkus update to

Re: [DISCUSS] KIP-1122: Create a dedicated data module for Kafka Connect data classes

2025-01-17 Thread Greg Harris
schema-converter/src/main/java/io/confluent/connect/schema/ConnectEnum.java [2] https://github.com/confluentinc/schema-registry/blob/e461e9659ef1532a95b3f2dfd38e843d766aa955/schema-converter/src/main/java/io/confluent/connect/schema/ConnectUnion.java [3] https://github.com/confluentinc/kafka-connect-

Re: [DISCUSS] KIP-1122: Create a dedicated data module for Kafka Connect data classes

2025-01-17 Thread Andrew Schofield
Hi Mario, Thanks for the KIP. My view is that it would be nice for Kafka to have a data model so that Kafka could do natively the things that people use schema registries for today. I'm not convinced that the Kafka Connect data classes are really up to the job, so I'm not sure that the

Re: [DISCUSS] KIP-1122: Create a dedicated data module for Kafka Connect data classes

2025-01-17 Thread Mario Fiore Vitale
omberg.net> ha scritto: >> > > >> > > > Hey Mario, >> > > > >> > > > From a quick read of this KIP, it's not super clear to me what >> problem >> > it >> > > > is trying to solve. The motivation states that &

[jira] [Resolved] (KAFKA-7776) Kafka Connect values converter parsing of ISO8601 not working properly

2025-01-16 Thread Andrew Schofield (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-7776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Schofield resolved KAFKA-7776. - Resolution: Fixed > Kafka Connect values converter parsing of ISO8601 not work

Re: [DISCUSS] KIP-1122: Create a dedicated data module for Kafka Connect data classes

2025-01-10 Thread Mario Fiore Vitale
I'm not sure what you mean > by > > > > that. > > > > > > > > Can you maybe add one or two examples of things that are not possible > > to > > > > do today and could be achieved by this refactor. If this proposal is > &

Re: [DISCUSS] KIP-1122: Create a dedicated data module for Kafka Connect data classes

2025-01-09 Thread Greg Harris
not sure what you mean by > > > that. > > > > > > Can you maybe add one or two examples of things that are not possible > to > > > do today and could be achieved by this refactor. If this proposal is > more > > > about improving the ergonomics of us

Re: [DISCUSS] KIP-1122: Create a dedicated data module for Kafka Connect data classes

2025-01-09 Thread Mario Fiore Vitale
refactor. If this proposal is more > > about improving the ergonomics of users of these classes, that's also > > valuable IMO. > > > > From: dev@kafka.apache.org At: 12/18/24 08:32:28 UTC-5:00To: > > dev@kafka.apache.org > > Subject: [DISCUSS] KIP-1122:

Re: [DISCUSS] KIP-1122: Create a dedicated data module for Kafka Connect data classes

2024-12-28 Thread Mario Fiore Vitale
his proposal is more > about improving the ergonomics of users of these classes, that's also > valuable IMO. > > From: dev@kafka.apache.org At: 12/18/24 08:32:28 UTC-5:00To: > dev@kafka.apache.org > Subject: [DISCUSS] KIP-1122: Create a dedicated data module for Kafka > Conn

Re:[DISCUSS] KIP-1122: Create a dedicated data module for Kafka Connect data classes

2024-12-20 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
SCUSS] KIP-1122: Create a dedicated data module for Kafka Connect data classes Hi Everyone, I would like to start a discussion on KIP-1122: Create a dedicated data module for Kafka Connect data classes [1]. [1] https://cwiki.apache.org/confluence/display/KAFKA/KIP-1122%3A+Create+a+dedicated +data

[DISCUSS] KIP-1122: Create a dedicated data module for Kafka Connect data classes

2024-12-18 Thread Mario Fiore Vitale
Hi Everyone, I would like to start a discussion on KIP-1122: Create a dedicated data module for Kafka Connect data classes [1]. [1] https://cwiki.apache.org/confluence/display/KAFKA/KIP-1122%3A+Create+a+dedicated+data+module+for+Kafka+Connect+data+classes Regards, -- Mario Fiore Vitale

[jira] [Created] (KAFKA-18299) Promote Kafka Connect data package as a dedicated module

2024-12-18 Thread Mario Fiore Vitale (Jira)
Mario Fiore Vitale created KAFKA-18299: -- Summary: Promote Kafka Connect data package as a dedicated module Key: KAFKA-18299 URL: https://issues.apache.org/jira/browse/KAFKA-18299 Project: Kafka

[jira] [Created] (KAFKA-18073) Data loss when Kafka Connect retriable exception raised

2024-11-22 Thread Tom Thornton (Jira)
Tom Thornton created KAFKA-18073: Summary: Data loss when Kafka Connect retriable exception raised Key: KAFKA-18073 URL: https://issues.apache.org/jira/browse/KAFKA-18073 Project: Kafka

[jira] [Created] (KAFKA-17982) Kafka Connect: Configurable JMX Port for Metrics Collection

2024-11-10 Thread Basapuram Kumar (Jira)
Basapuram Kumar created KAFKA-17982: --- Summary: Kafka Connect: Configurable JMX Port for Metrics Collection Key: KAFKA-17982 URL: https://issues.apache.org/jira/browse/KAFKA-17982 Project: Kafka

[jira] [Created] (KAFKA-17958) Publish metrics for skipped records in Kafka Connect

2024-11-07 Thread Hector Geraldino (Jira)
Hector Geraldino created KAFKA-17958: Summary: Publish metrics for skipped records in Kafka Connect Key: KAFKA-17958 URL: https://issues.apache.org/jira/browse/KAFKA-17958 Project: Kafka

[jira] [Resolved] (KAFKA-17916) Convert Kafka Connect system tests to use KRaft

2024-11-05 Thread Kevin Wu (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-17916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Wu resolved KAFKA-17916. -- Resolution: Fixed > Convert Kafka Connect system tests to use KR

[jira] [Created] (KAFKA-17916) Convert Kafka Connect system tests to use KRaft

2024-10-31 Thread Kevin Wu (Jira)
Kevin Wu created KAFKA-17916: Summary: Convert Kafka Connect system tests to use KRaft Key: KAFKA-17916 URL: https://issues.apache.org/jira/browse/KAFKA-17916 Project: Kafka Issue Type

[jira] [Created] (KAFKA-17424) Memory optimisation for Kafka-connect

2024-08-26 Thread Ajit Singh (Jira)
Ajit Singh created KAFKA-17424: -- Summary: Memory optimisation for Kafka-connect Key: KAFKA-17424 URL: https://issues.apache.org/jira/browse/KAFKA-17424 Project: Kafka Issue Type: Improvement

[jira] [Resolved] (KAFKA-12818) Memory leakage when kafka connect 2.7 uses directory config provider

2024-08-23 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-12818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-12818. - Resolution: Duplicate > Memory leakage when kafka connect 2.7 uses directory config provi

Re: [DISCUSS] KIP-802: Validation Support for Kafka Connect SMT Options

2024-07-29 Thread Mickael Maison
Hi, I've not received a reply from Gunnar since last month, so I'll pick this KIP up. Thanks, Mickael On Tue, Jun 18, 2024 at 6:01 PM Mickael Maison wrote: > > Hi Gunnar, > > I think this KIP would be a great addition to Kafka Connect but it > looks like it'

Re: [VOTE] KIP-1017: A health check endpoint for Kafka Connect

2024-06-20 Thread Chris Egerton
gt; > Regards, > > - Adrian > > > > From: Edoardo Comar > > Date: Sunday, 16 June 2024 at 07:57 > > To: dev@kafka.apache.org > > Subject: [EXTERNAL] Re: [VOTE] KIP-1017: A health check endpoint for > Kafka Connect > > Thanks for this KIP, Chris, &

Re: [DISCUSS] KIP-802: Validation Support for Kafka Connect SMT Options

2024-06-18 Thread Mickael Maison
Hi Gunnar, I think this KIP would be a great addition to Kafka Connect but it looks like it's been abandoned. Are you still interested in working on this? If you need some time or help, that's fine, just let us know. If not, no worries, I'm happy to pick it up if needed. Thanks,

[jira] [Resolved] (KAFKA-12164) ssue when kafka connect worker pod restart, during creation of nested partition directories in hdfs file system.

2024-06-17 Thread Greg Harris (Jira)
supported by the Apache Kafka project. If/when an issue with the Connect framework is implicated, a new ticket may be opened with details about that issue. > ssue when kafka connect worker pod restart, during creation of nested > partition directories in hdfs file

Re: [VOTE] KIP-1017: A health check endpoint for Kafka Connect

2024-06-17 Thread Federico Valeri
t: [EXTERNAL] Re: [VOTE] KIP-1017: A health check endpoint for Kafka > Connect > Thanks for this KIP, Chris, > > +1 (binding) > > Edo > > On Fri, 14 Jun 2024 at 20:20, Greg Harris > wrote: > > > > Hey Chris, > > > > Thanks for the KIP. > > > &

RE: [VOTE] KIP-1017: A health check endpoint for Kafka Connect

2024-06-17 Thread Adrian Preston
Hi Chris, +1 (non-binding) Regards, - Adrian From: Edoardo Comar Date: Sunday, 16 June 2024 at 07:57 To: dev@kafka.apache.org Subject: [EXTERNAL] Re: [VOTE] KIP-1017: A health check endpoint for Kafka Connect Thanks for this KIP, Chris, +1 (binding) Edo On Fri, 14 Jun 2024 at 20:20, Greg

Re: [VOTE] KIP-1017: A health check endpoint for Kafka Connect

2024-06-15 Thread Edoardo Comar
4 at 8:44 AM Hector Geraldino (BLOOMBERG/ 919 3RD A) < > hgerald...@bloomberg.net> wrote: > > > Thanks Chris, > > > > +1 (non-binding) > > > > From: dev@kafka.apache.org At: 06/14/24 11:43:02 UTC-4:00To: > > dev@kafka.apache.org > > Subj

Re: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

2024-06-15 Thread Chris Egerton
t; > > itself as a log about the workers' behavior. I understand if you > think > > > > it's > > > > > such a scope change that it should be an improvement KIP, but would > > > like > > > > to > > > > > gauge wha

Re: [VOTE] KIP-1017: A health check endpoint for Kafka Connect

2024-06-14 Thread Greg Harris
At: 06/14/24 11:43:02 UTC-4:00To: > dev@kafka.apache.org > Subject: Re: [VOTE] KIP-1017: A health check endpoint for Kafka Connect > > Hi Chris, > > +1 (binding) > > Thanks, > Mickael > > On Fri, Jun 14, 2024 at 5:01 PM Andrew Schofield > wrote: > >

Re: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

2024-06-14 Thread Edoardo Comar
scope change that it should be an improvement KIP, but would > > like > > > to > > > > gauge what you think about this. > > > > > > > > Regards, > > > > Viktor > > > > > > > > On Tue, Jun 11, 2024 at 4:34 PM Chris Eg

Re: [VOTE] KIP-1017: A health check endpoint for Kafka Connect

2024-06-14 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Thanks Chris, +1 (non-binding) From: dev@kafka.apache.org At: 06/14/24 11:43:02 UTC-4:00To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-1017: A health check endpoint for Kafka Connect Hi Chris, +1 (binding) Thanks, Mickael On Fri, Jun 14, 2024 at 5:01 PM Andrew Schofield wrote: > &

Re: [VOTE] KIP-1017: A health check endpoint for Kafka Connect

2024-06-14 Thread Mickael Maison
> > > Happy Friday! I'd like to kick off a vote on KIP-1017. > > > > Design doc: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1017%3A+Health+check+endpoint+for+Kafka+Connect > > > > Discussion thread: > > https://lists.apache.org/thread/95nto84wtogdgk3v97rxcwxr258wnjnt > > > > Cheers, > > > > Chris >

Re: [VOTE] KIP-1017: A health check endpoint for Kafka Connect

2024-06-14 Thread Andrew Schofield
Hi Chris, Thanks for the KIP. +1 (non-binding) Thanks, Andrew > On 14 Jun 2024, at 15:48, Chris Egerton wrote: > > Hi all, > > Happy Friday! I'd like to kick off a vote on KIP-1017. > > Design doc: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1017%3A+He

[VOTE] KIP-1017: A health check endpoint for Kafka Connect

2024-06-14 Thread Chris Egerton
Hi all, Happy Friday! I'd like to kick off a vote on KIP-1017. Design doc: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1017%3A+Health+check+endpoint+for+Kafka+Connect Discussion thread: https://lists.apache.org/thread/95nto84wtogdgk3v97rxcwxr258wnjnt Cheers, Chris

Re: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

2024-06-14 Thread Viktor Somogyi-Vass
ed. I don't plan on > > > adding any structured data such as error codes or something like a > > "phase" > > > field with values like READING_CONFIG_TOPIC quite yet, but there is > room > > > for us to add human-readable information on t

Re: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

2024-06-14 Thread Chris Egerton
t; Thanks for your comments/questions! The responses to them are related > > so > > > > I'll try to address both at once. > > > > > > > > The most recent update I made to the KIP should help provide insight > > into > > > > what's

Re: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

2024-06-11 Thread Chris Egerton
room > > for us to add human-readable information on the causes of failure in the > > "message" field (see KAFKA-15563 [1] and its PR [2] for an example of > what > > kind of information we might provide to users). Part of the problem is > that > > while

Re: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

2024-06-11 Thread Viktor Somogyi-Vass
ation on the causes of failure in the > "message" field (see KAFKA-15563 [1] and its PR [2] for an example of what > kind of information we might provide to users). Part of the problem is that > while I've heard plenty of (justified!) complaints about the Kafka Connect

RE: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

2024-06-11 Thread Adrian Preston
robust implementation before considering further enhancements. Cheers, - Adrian. From: Chris Egerton Date: Tuesday, 11 June 2024 at 15:34 To: dev@kafka.apache.org Subject: [EXTERNAL] Re: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect Hi Adrian, Thanks for your comments/questions

Re: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

2024-06-11 Thread Chris Egerton
ind of information we might provide to users). Part of the problem is that while I've heard plenty of (justified!) complaints about the Kafka Connect REST API becoming unavailable and the difficulties users face with debugging their workers when that happens, I still don't feel we have a

RE: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

2024-06-11 Thread Adrian Preston
) transition more of the conditions over to being handled by automation. Regards, - Adrian From: Chris Egerton Date: Monday, 10 June 2024 at 15:26 To: dev@kafka.apache.org Subject: [EXTERNAL] Re: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect Hi all, Thanks for the positive feedback

Re: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

2024-06-10 Thread Chris Egerton
e KIP looks good to me. +1 > > > > Thanks, > > Mickael > > > > On Fri, Jan 26, 2024 at 8:41 PM Chris Egerton > wrote: > >> > >> Hi all, > >> > >> Happy Friday! I'd like to kick off discussion for KIP-1017, which (as > the >

Re: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

2024-06-07 Thread Andrew Schofield
KIP looks good to me. +1 > > Thanks, > Mickael > > On Fri, Jan 26, 2024 at 8:41 PM Chris Egerton wrote: >> >> Hi all, >> >> Happy Friday! I'd like to kick off discussion for KIP-1017, which (as the >> title suggests) proposes adding a health check e

Re: [DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

2024-06-07 Thread Mickael Maison
Hi Chris, Happy Friday! The KIP looks good to me. +1 Thanks, Mickael On Fri, Jan 26, 2024 at 8:41 PM Chris Egerton wrote: > > Hi all, > > Happy Friday! I'd like to kick off discussion for KIP-1017, which (as the > title suggests) proposes adding a health check endpoi

[jira] [Resolved] (KAFKA-16838) Kafka Connect loads old tasks from removed connectors

2024-06-04 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-16838. --- Fix Version/s: 3.9.0 Resolution: Fixed > Kafka Connect loads old tasks from remo

[jira] [Resolved] (KAFKA-16837) Kafka Connect fails on update connector for incorrect previous Config Provider tasks

2024-06-04 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-16837. --- Fix Version/s: 3.9.0 Resolution: Fixed > Kafka Connect fails on update connector

[jira] [Resolved] (KAFKA-5451) Kafka Connect should scan classpath asynchronously

2024-05-28 Thread Greg Harris (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-5451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Harris resolved KAFKA-5451. Resolution: Won't Do Kafka Connect needs the results of plugin scanning to answer basic

[jira] [Resolved] (KAFKA-6208) Reduce startup time for Kafka Connect workers

2024-05-28 Thread Greg Harris (Jira)
=service_load on 3.6+, see KAFKA-14627/KIP-898 for more details. > Reduce startup time for Kafka Connect workers > - > > Key: KAFKA-6208 > URL: https://issues.apache.org/jira/browse/KAFKA-6208 >

[jira] [Created] (KAFKA-16838) Kafka Connect loads old tasks from removed connectors

2024-05-24 Thread Sergey Ivanov (Jira)
Sergey Ivanov created KAFKA-16838: - Summary: Kafka Connect loads old tasks from removed connectors Key: KAFKA-16838 URL: https://issues.apache.org/jira/browse/KAFKA-16838 Project: Kafka

[jira] [Created] (KAFKA-16837) Kafka Connect fails on update connector for incorrect previous Config Provider tasks

2024-05-24 Thread Sergey Ivanov (Jira)
Sergey Ivanov created KAFKA-16837: - Summary: Kafka Connect fails on update connector for incorrect previous Config Provider tasks Key: KAFKA-16837 URL: https://issues.apache.org/jira/browse/KAFKA-16837

[jira] [Resolved] (KAFKA-16603) Data loss when kafka connect sending data to Kafka

2024-05-20 Thread Chris Egerton (Jira)
[ https://issues.apache.org/jira/browse/KAFKA-16603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Egerton resolved KAFKA-16603. --- Resolution: Not A Bug > Data loss when kafka connect sending data to Ka

[jira] [Created] (KAFKA-16603) Data loss when kafka connect sending data to Kafka

2024-04-22 Thread Anil Dasari (Jira)
Anil Dasari created KAFKA-16603: --- Summary: Data loss when kafka connect sending data to Kafka Key: KAFKA-16603 URL: https://issues.apache.org/jira/browse/KAFKA-16603 Project: Kafka Issue Type

[jira] [Created] (KAFKA-16326) Kafka Connect unable to find javax dependency on Quarkus update to 3.X

2024-03-04 Thread Pau Ortega Puig (Jira)
Pau Ortega Puig created KAFKA-16326: --- Summary: Kafka Connect unable to find javax dependency on Quarkus update to 3.X Key: KAFKA-16326 URL: https://issues.apache.org/jira/browse/KAFKA-16326 Project

Re: [DISCUSS] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2024-02-15 Thread Taras Ledkov
what seems like a simple and necessary patch. Is the SslEngineFactory refactoring requirement blocking? [1]. https://cwiki.apache.org/confluence/display/KAFKA/KIP-967%3A+Support+custom+SSL+configuration+for+Kafka+Connect+RestServer [2]. https://lists.apache.org/thread/wc4v5v3pynl15g1q547m2croqsqgmzpw

Re: [DISCUSS] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2024-02-15 Thread Taras Ledkov
https://cwiki.apache.org/confluence/display/KAFKA/KIP-967%3A+Support+custom+SSL+configuration+for+Kafka+Connect+RestServer [2]. https://lists.apache.org/thread/wc4v5v3pynl15g1q547m2croqsqgmzpw -- With best regards, Taras Ledkov

[DISCUSS] KIP-1017: A health check endpoint for Kafka Connect

2024-01-26 Thread Chris Egerton
Hi all, Happy Friday! I'd like to kick off discussion for KIP-1017, which (as the title suggests) proposes adding a health check endpoint for Kafka Connect: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1017%3A+Health+check+endpoint+for+Kafka+Connect This is one of the longest-sta

Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect

2024-01-11 Thread Chris Egerton
> > Thanks for the KIP! I think the aggressive default and deprecation > > > > > schedule is the right choice for this change. > > > > > > > > > > +1 (binding) > > > > > > > > > > On Wed, Jan 3, 2024 at 9:01 AM Mickael

Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect

2024-01-05 Thread Greg Harris
d, Jan 3, 2024 at 9:01 AM Mickael Maison < > > mickael.mai...@gmail.com> > > > > wrote: > > > > > > > > > > Hi Chris, > > > > > > > > > > +1 (binding), thanks for the KIP > > > > > > > > > >

Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect

2024-01-05 Thread Chris Egerton
; > Mickael > > > > > > > > On Tue, Jan 2, 2024 at 8:55 PM Hector Geraldino (BLOOMBERG/ 919 3RD > A) > > > > wrote: > > > > > > > > > > +1 (non-binding) > > > > > > > > > > Th

Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect

2024-01-04 Thread Federico Valeri
nding), thanks for the KIP > > > > > > Mickael > > > > > > On Tue, Jan 2, 2024 at 8:55 PM Hector Geraldino (BLOOMBERG/ 919 3RD A) > > > wrote: > > > > > > > > +1 (non-binding) > > > > > > > > Thanks Chris! >

Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect

2024-01-03 Thread Yash Mayya
+1 (binding), thanks for the KIP > > > > Mickael > > > > On Tue, Jan 2, 2024 at 8:55 PM Hector Geraldino (BLOOMBERG/ 919 3RD A) > > wrote: > > > > > > +1 (non-binding) > > > > > > Thanks Chris! > > > > > > From: dev@

Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect

2024-01-03 Thread Greg Harris
, 2024 at 8:55 PM Hector Geraldino (BLOOMBERG/ 919 3RD A) > wrote: > > > > +1 (non-binding) > > > > Thanks Chris! > > > > From: dev@kafka.apache.org At: 01/02/24 11:49:18 UTC-5:00To: > > dev@kafka.apache.org > > Subject: Re: [VOTE] KIP-1004: En

Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect

2024-01-03 Thread Mickael Maison
ect: Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect > > Hi all, > > Happy New Year! Wanted to give this a bump now that the holidays are over > for a lot of us. Looking forward to people's thoughts! > > Cheers, > > Chris > > On Mon, Dec 4, 2023 at 10

Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect

2024-01-02 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
+1 (non-binding) Thanks Chris! From: dev@kafka.apache.org At: 01/02/24 11:49:18 UTC-5:00To: dev@kafka.apache.org Subject: Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect Hi all, Happy New Year! Wanted to give this a bump now that the holidays are over for a lot of us

Re: [VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect

2024-01-02 Thread Chris Egerton
enforcement for the > tasks.max connector property in Kafka Connect. > > The KIP: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1004%3A+Enforce+tasks.max+property+in+Kafka+Connect > > The discussion thread: > https://lists.apache.org/thread/scx75cjwm19jyt19wxky41q9smf5nx6d > > Cheers, > > Chris >

Re: [DISCUSS] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2023-12-20 Thread Taras Ledkov
Hi team, Ping for review / vote for KIP-967 [1]. Voting thread is here [2] [1]. https://cwiki.apache.org/confluence/display/KAFKA/KIP-967%3A+Support+custom+SSL+configuration+for+Kafka+Connect+RestServer [2]. https://lists.apache.org/thread/wc4v5v3pynl15g1q547m2croqsqgmzpw -- With best regards

[DISCUSS] Kafka Connect source task interruption semantics

2023-12-12 Thread Chris Egerton
entally introduced a breaking change for Kafka Connect in https://github.com/apache/kafka/pull/9669. Before that change, the SourceTask::stop method [1] would be invoked on a separate thread from the one that did the actual data processing for the task (polling the task for records, transforming

[jira] [Created] (KAFKA-15988) Kafka Connect OffsetsApiIntegrationTest takes too long

2023-12-07 Thread Chris Egerton (Jira)
Chris Egerton created KAFKA-15988: - Summary: Kafka Connect OffsetsApiIntegrationTest takes too long Key: KAFKA-15988 URL: https://issues.apache.org/jira/browse/KAFKA-15988 Project: Kafka

Re: [DISCUSS] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2023-12-06 Thread Taras Ledkov
are being passed to their factory. Instead, the default SSL > > engine factory used by Connect can perform this logic, and we can let other > > custom factories be responsible for their own default values. > > > > > > Cheers, > > > > Chris > > > &

Re: [DISCUSS] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2023-12-06 Thread Taras Ledkov
ed, Nov 29, 2023 at 8:36 AM Taras Ledkov wrote: > > > Hi team, > > > > Ping for review / vote for KIP-967 [1]. > > Voting thread is here [2] > > > > [1]. > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-967%3A+Support+custom+SSL+configuration+f

Re: [DISCUSS] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2023-12-04 Thread Greg Harris
gt; Hi team, > > > > Ping for review / vote for KIP-967 [1]. > > Voting thread is here [2] > > > > [1]. > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-967%3A+Support+custom+SSL+configuration+for+Kafka+Connect+RestServer > > [2]. https://github.com/apache/kafka/pull/14203 > > [2]. https://lists.apache.org/thread/wc4v5v3pynl15g1q547m2croqsqgmzpw > > > > -- > > With best regards, > > Taras Ledkov > >

Re: [DISCUSS] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2023-12-04 Thread Chris Egerton
, Nov 29, 2023 at 8:36 AM Taras Ledkov wrote: > Hi team, > > Ping for review / vote for KIP-967 [1]. > Voting thread is here [2] > > [1]. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-967%3A+Support+custom+SSL+configuration+for+Kafka+Connect+RestServer > [2]

Re: [DISCUSS] KIP-1004: Enforce tasks.max property in Kafka Connect

2023-12-04 Thread Chris Egerton
e Public Interfaces > > > > section states: > > > > > > > > "If the connector generated excessive tasks after being reconfigured, > > > then > > > > any existing tasks for the connector will be allowed to continue > > runn

[VOTE] KIP-1004: Enforce tasks.max property in Kafka Connect

2023-12-04 Thread Chris Egerton
Hi all, I'd like to call for a vote on KIP-1004, which adds enforcement for the tasks.max connector property in Kafka Connect. The KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1004%3A+Enforce+tasks.max+property+in+Kafka+Connect The discussion thread: https://lists.apach

Re: [DISCUSS] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2023-11-29 Thread Taras Ledkov
Hi team, Ping for review / vote for KIP-967 [1]. Voting thread is here [2] [1]. https://cwiki.apache.org/confluence/display/KAFKA/KIP-967%3A+Support+custom+SSL+configuration+for+Kafka+Connect+RestServer [2]. https://github.com/apache/kafka/pull/14203 [2]. https://lists.apache.org/thread

Re: [DISCUSS] KIP-1004: Enforce tasks.max property in Kafka Connect

2023-11-24 Thread Chris Egerton
ed to continue > running, > > > unless that existing set of tasks also exceeds the tasks.max property." > > > > > > Would not failing the connector land us in the second scenario of > > > 'Rejected Alternatives'? > > > > > > From

Re: [DISCUSS] KIP-1004: Enforce tasks.max property in Kafka Connect

2023-11-22 Thread Yash Mayya
asks for the connector will be allowed to continue running, > > unless that existing set of tasks also exceeds the tasks.max property." > > > > Would not failing the connector land us in the second scenario of > > 'Rejected Alternatives'? > > > > Fr

Re: [DISCUSS] KIP-1004: Enforce tasks.max property in Kafka Connect

2023-11-20 Thread Chris Egerton
ev@kafka.apache.org > Subject: [DISCUSS] KIP-1004: Enforce tasks.max property in Kafka Connect > > Hi all, > > I'd like to open up KIP-1004 for discussion: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1004%3A+Enforce+tasks.max+ > property+in+Kafka+Connect > &g

Re:[DISCUSS] KIP-1004: Enforce tasks.max property in Kafka Connect

2023-11-20 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
S] KIP-1004: Enforce tasks.max property in Kafka Connect Hi all, I'd like to open up KIP-1004 for discussion: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1004%3A+Enforce+tasks.max+ property+in+Kafka+Connect As a brief summary: this KIP proposes that the Kafka Connect runtime start failing c

[jira] [Created] (KAFKA-15841) Add Support for Topic-Level Partitioning in Kafka Connect

2023-11-16 Thread Henrique Mota (Jira)
Henrique Mota created KAFKA-15841: - Summary: Add Support for Topic-Level Partitioning in Kafka Connect Key: KAFKA-15841 URL: https://issues.apache.org/jira/browse/KAFKA-15841 Project: Kafka

[DISCUSS] KIP-1004: Enforce tasks.max property in Kafka Connect

2023-11-10 Thread Chris Egerton
Hi all, I'd like to open up KIP-1004 for discussion: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1004%3A+Enforce+tasks.max+property+in+Kafka+Connect As a brief summary: this KIP proposes that the Kafka Connect runtime start failing connectors that generate a greater number of

Re: [DISCUSS] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2023-11-07 Thread Taras Ledkov
y? Let me know what you think. All I can think of is: - BaseSslEngineFactory; - StaticSslEngineFactory; - NonReconfigurableSslEngineFactory. But I'm afraid that it would be hard to find sponsors to review this. My proposal: restrict the scope of this KIP to Kafka Connect config

Re: [DISCUSS] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2023-11-02 Thread Chris Egerton
actory may want to use property names that aren't available out-of-the-box with Kafka Connect (i.e., which aren't defined in the DistributedConfig [5] or StandaloneConfig [6] class). It also sounds like (reading the KIP again) we're proposing that SSL engine factory classes only be c

Re: [DISCUSS] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2023-11-02 Thread Taras Ledkov
ies are not prefixed with "listeners.https.". I guess that the definitions of SSL properties at the root of the `WorkerConfig` is unclean / buggy / may lead to unexpected behavior. So, I would like to fix / improve this test. > 5. There are several methods in the SslEngineFactory interfa

Re: [DISCUSS] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2023-10-30 Thread Chris Egerton
ests that are already covered by the existing code base, but if that's the case, it'd be nice to see this clarified as the assumption is usually that items in the test plan cover changes that will have to be implemented for the KIP. 5. There are several methods in the Ssl

Re: [VOTE] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2023-10-26 Thread Николай Ижиков
II'd like to call a vote on KIP-967: Support custom SSL configuration for >> Kafka Connect RestServer [1]. >> Discussion thread [2] was started more then 2 month ago and there was not >> any negative or critical comments. >> >> [1]. >> https://cwik

Re: [VOTE] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2023-10-23 Thread Greg Harris
nding) Thanks, Greg On Fri, Oct 20, 2023 at 8:03 AM Taras Ledkov wrote: > > Hi Kafka Team. > > II'd like to call a vote on KIP-967: Support custom SSL configuration for > Kafka Connect RestServer [1]. > Discussion thread [2] was started more then 2 month ago and ther

[VOTE] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2023-10-20 Thread Taras Ledkov
Hi Kafka Team. II'd like to call a vote on KIP-967: Support custom SSL configuration for Kafka Connect RestServer [1]. Discussion thread [2] was started more then 2 month ago and there was not any negative or critical comments. [1]. https://cwiki.apache.org/confluence/display/KAFKA/KIP-9

Re:[DISCUSS] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2023-10-12 Thread Taras Ledkov
Hi Ashwin, > I was referring to (and did not understand) the removal of L141 in > clients/src/main/java/org/apache/kafka/common/security/ssl/SslFactory.java This line is moved to "new" private method `instantiateSslEngineFactory0 `. Please take a look at the `SslFactory:L132` at the patch. Just

Re: [VOTE] KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-10-11 Thread Chris Egerton
as well > > De: dev@kafka.apache.org A: 10/09/23 05:16:06 UTC-4:00A: > dev@kafka.apache.org > Subject: Re: [VOTE] KIP-976: Cluster-wide dynamic log adjustment for Kafka > Connect > > Hi Chris, > > +1 (non binding) > > Thanks > Fede > > On Sun, Oct 8, 2023 at

Re: [DISCUSS] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2023-10-09 Thread Ashwin
Hello Taras, > Do you think that something needs to be corrected in KIP to make it more understandable without PR? Do you have any advice? Ha - no. I just wanted to thank you for sharing the PR which helped me as a newbie. > If I understood the question correctly: I was referring to (and did not

Re: [VOTE] KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-10-09 Thread Hector Geraldino (BLOOMBERG/ 919 3RD A)
Good stuff, +1 (non-binding) from me as well De: dev@kafka.apache.org A: 10/09/23 05:16:06 UTC-4:00A: dev@kafka.apache.org Subject: Re: [VOTE] KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect Hi Chris, +1 (non binding) Thanks Fede On Sun, Oct 8, 2023 at 10:11 AM Yash Mayya

Re: Re: [VOTE] KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-10-09 Thread hudeqi
Hi Chris, +1 (non-binding) Finally, there is no need to use external intrusion tools to change the log level of kafka connect online! Thanks for the KIP! best, hudeqi

Re: [VOTE] KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-10-09 Thread Mickael Maison
2023 at 9:05 AM Chris Egerton > > > wrote: > > > > > > > > Hi all, > > > > > > > > I'd like to call for a vote on KIP-976, which augments the existing > > > dynamic > > > > logger adjustment REST API for

Re:[DISCUSS] KIP-967: Support custom SSL configuration for Kafka Connect RestServer

2023-10-09 Thread Taras Ledkov
Hi, Kafka team. 1. Ping to review KIP. 2. I dare say that the low activity in the discussion of KIP-967 means that KIP-967 is ready for voting? -- With best regards Taras Ledkov

Re: [VOTE] KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-10-09 Thread Federico Valeri
gt; > > > On Fri, Oct 6, 2023 at 9:05 AM Chris Egerton > > wrote: > > > > > > Hi all, > > > > > > I'd like to call for a vote on KIP-976, which augments the existing > > dynamic > > > logger adjustment REST API for Kafka Connect

Re: Kafka Connect - Customize REST request headers

2023-10-09 Thread Yeikel Santana
Authorization = headers.getHeaderString(HttpHeaders.AUTHORIZATION);     if (credentialAuthorization != null) {     req.header(HttpHeaders.AUTHORIZATION, credentialAuthorization);     }     }     } This is of course risky and it would be significantly more convenient if this functi

Re: [VOTE] KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-10-08 Thread Yash Mayya
> broadcast is better than REST forwarding. > > +1 (binding) > > Thanks, > Greg > > On Fri, Oct 6, 2023 at 9:05 AM Chris Egerton > wrote: > > > > Hi all, > > > > I'd like to call for a vote on KIP-976, which augments the existing > dynamic &

Re: Kafka Connect - Customize REST request headers

2023-10-06 Thread Chris Egerton
Hi Yeikel, Neat question! And thanks for the link to the RestClient code; very helpful. I don't believe there's a way to configure Kafka Connect to add these headers to forwarded requests right now. You may be able to do some kind of out-of-band proxy magic to intercept forwarded re

Re: [VOTE] KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-10-06 Thread Greg Harris
: > > Hi all, > > I'd like to call for a vote on KIP-976, which augments the existing dynamic > logger adjustment REST API for Kafka Connect to apply changes cluster-wide > instead on a per-worker basis. > > The KIP: > https://cwiki.apache.org/confluence/display/KAFKA/K

[VOTE] KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect

2023-10-06 Thread Chris Egerton
Hi all, I'd like to call for a vote on KIP-976, which augments the existing dynamic logger adjustment REST API for Kafka Connect to apply changes cluster-wide instead on a per-worker basis. The KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-976:+Cluster-wide+dynamic+log+adjus

  1   2   3   4   5   6   7   8   9   10   >