Re: [DISCUSS] KIP-381 Connect: Tell about records that had their offsets flushed in callback

2018-10-15 Thread Konstantine Karantasis
This is a significant improvement to the semantics of source connectors. I'm expecting that it will facilitate source connector implementations and even enrich the application uses cases that we see. I only have a few minor suggestions at the moment. I believe that Acked is a common abbreviation f

Re: [EXTERNAL] Incremental Cooperative Rebalancing

2018-10-16 Thread Konstantine Karantasis
esign#Kafka0.9ConsumerRewriteDesign-Interestingscenariostoconsider > [4] > > https://cwiki.apache.org/confluence/display/KAFKA/Incremental+Cooperative+Rebalancing+for+Streams > > > On Thu, Oct 4, 2018 at 12:16 PM, McCaig, Rhys > wrote: > > > This is fantast

Re: [DISCUSS] KIP-381 Connect: Tell about records that had their offsets flushed in callback

2018-10-17 Thread Konstantine Karantasis
ally understands what this is about. Some > times you only really understand what something is about, when you are > forced to write about it (at least that is my excuse ). > > Regards, Per Steffensen > > On 16/10/2018 05.57, Konstantine Karantasis wrote: > > This is a significa

Re: [DISCUSS] KIP-345: Reduce multiple consumer rebalances by specifying member id

2018-11-20 Thread Konstantine Karantasis
Hi Boyang. Thanks for preparing this KIP! It is making good progress and will be a great improvement for stateful Kafka applications. Apologies for my late reply, I was away for a while. Lots of great comments so far, so I'll probably second most of them in what I suggest below at this point. Wh

Re: [DISCUSS] KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request

2018-12-14 Thread Konstantine Karantasis
Hi Alex, thanks for working on this and identifying the need to address this issue. I see under Rejected Alternatives that the option to disable this feature has been considered already. However, I'd like to return a bit to this option and highlight the following: * Exposing this request was not

Re: [DISCUSS] KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request

2018-12-17 Thread Konstantine Karantasis
Thanks for considering removal Alex. I totally agree with your assessment. Still, I'd be in favor of making KIP-404 a small KIP that describes that this option is now being disabled. (If I'm not mistaken, one place I've noticed this feature being used is in Connect's unit tests for the rest interfa

Re: [DISCUSS] KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request

2018-12-18 Thread Konstantine Karantasis
is proposed to be > introduced in the KIP, > it was never used anywhere else than in unit tests, as a part of PR for the > KIP. > So no public interface changes are needed if we proceed without exposing > the option. > WDYT? > > Regards, Alex. > > On Mon,

Re: [DISCUSS] KIP-404: Add Kafka Connect configuration parameter for disabling WADL output on OPTIONS request

2018-12-19 Thread Konstantine Karantasis
is > > as a PR and backporting -- the WADL operation was never documented nor > > described in any of the prior KIPs, and it seems like an implementation > > detail that leaked out accidentally. > > > > Randall > > > > On Tue, Dec 18, 2018 at 4:15 PM Konstant

Re: [DISCUSS] KIP-411: Add option to make Kafka Connect task client ID values unique

2019-01-04 Thread Konstantine Karantasis
Hi Paul. I second Ewen and I intended to give similar feedback: 1) Can we avoid a config altogether? 2) If we prefer to add a config anyways, can we use a set of allowed values instead of a boolean, even if initially these values are only two? As the discussion on Jira highlights, there is a pote

[DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-01-11 Thread Konstantine Karantasis
Hi all, I just published KIP-415: Incremental Cooperative Rebalancing in Kafka Connect on the wiki here: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals This is the first KIP to suggest an implementation of incremental and cooperative rebalancing in the context of Ka

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-01-17 Thread Konstantine Karantasis
a > previous assignment) > Have we decided on whether we would make use of the optimization as to not > send the assignment that the worker already knows about? > > I enjoyed reading the rebalancing examples. As a small readability > improvement, could I suggest we clarify which Wo

Re: [DISCUSS] KIP-298: Error Handling in Connect

2018-05-16 Thread Konstantine Karantasis
Arjun, it's exciting to see a KIP around better handling of bad-data and errors in Kafka Connect. I have only a few comments below, which I hope you'll find helpful. 1. I think it'd be useful to describe a bit more in detail how someone can extract the raw data of a Kafka record that failed to ge

Re: [DISCUSS] KIP-298: Error Handling in Connect

2018-05-17 Thread Konstantine Karantasis
Thanks Arjun for your quick response. Adding an example for the failure log improves things, but I think it'd be better to also add the schema definition of these Json entries. And I'll agree with Magesh that this format should be public API. Also, does the current example have a copy/paste typo?

Re: [VOTE] KIP-285: Connect Rest Extension Plugin

2018-05-18 Thread Konstantine Karantasis
+1 (non-binding) - Konstantine On Thu, May 17, 2018 at 10:05 PM, Ewen Cheslack-Postava wrote: > +1 (binding) > > Thanks, > Ewen > > On Thu, May 17, 2018 at 12:16 PM Ted Yu wrote: > > > +1 > > Original message From: Gwen Shapira > > Date: 5/17/18 12:02 PM (GMT-08:00) To: dev

Re: [DISCUSS] KIP-298: Error Handling in Connect

2018-05-18 Thread Konstantine Karantasis
cause we're rendering > it > > as JSON. Not sure that's a good idea... > > > > I think that last item might be the biggest concern to me -- DLQ formats > > and control over content & reprocessing seems a bit unclear to me here, > so > > I'd assume user

Re: [VOTE] KIP-298: Error Handling in Connect kafka

2018-05-18 Thread Konstantine Karantasis
+1 (non-binding) - Konstantine On Thu, May 17, 2018 at 12:58 AM, Arjun Satish wrote: > All, > > Many thanks for all the feedback on KIP-298. Highly appreciate the time and > effort you all put into it. > > I've updated the KIP accordingly, and would like to start to start a vote > on it. > > KI

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

2018-06-11 Thread Konstantine Karantasis
Hi everyone, after fixing an issue with a regular expression in Connect's class loading isolation of the new component type ConfigProvider here: https://github.com/apache/kafka/pull/5177 I noticed that the new interface ConfigProvider, along with its first implementation FileConfigProvider, have

Re: [DISCUSS] KIP-297: Externalizing Secrets for Connect Configurations

2018-06-11 Thread Konstantine Karantasis
gt; > Sounds reasonable to me too. > > > > Regards, > > > > Rajini > > > > On Mon, Jun 11, 2018 at 7:55 PM, Robert Yokota > wrote: > > > > > Hi Konstantine, > > > > > > Sounds reasonable! > > > > > > Tha

Re: [ANNOUNCE] New committer: Matthias J. Sax

2018-01-15 Thread Konstantine Karantasis
Matthias! Congratulations! Konstantine On Mon, Jan 15, 2018 at 4:18 AM, Edoardo Comar wrote: > Congratulations Matthias ! > -- > > Edoardo Comar > > IBM Message Hub > > IBM UK Ltd, Hursley Park, SO21 2JN > > > > From: UMESH CHAUDHARY > To:

Re: [ANNOUNCE] New Kafka PMC Member: Rajini Sivaram

2018-01-17 Thread Konstantine Karantasis
Congrats Rajini! -Konstantine On Wed, Jan 17, 2018 at 2:18 PM, Becket Qin wrote: > Congratulations, Rajini! > > On Wed, Jan 17, 2018 at 1:52 PM, Ismael Juma wrote: > > > Congratulations Rajini! > > > > On 17 Jan 2018 10:49 am, "Gwen Shapira" wrote: > > > > Dear Kafka Developers, Users and Fan

Re: [VOTE] KIP-145: Expose Record Headers in Kafka Connect

2018-01-22 Thread Konstantine Karantasis
Great addition! +1 (non-binding) Konstantine On Sun, Jan 21, 2018 at 7:26 PM, Ewen Cheslack-Postava wrote: > +1 (binding) > > Thanks for the work on this -- not a small upgrade to the Connect APIs! > > -Ewen > > On Fri, Jan 19, 2018 at 3:37 PM, Randall Hauch wrote: > > > Hi everyone, > > > >

Re: [VOTE] KIP-212: Enforce set of legal characters for connector names

2018-01-23 Thread Konstantine Karantasis
Thanks for the KIP! +1 (non-binding) Konstantine On Tue, Jan 23, 2018 at 10:48 AM, Ewen Cheslack-Postava wrote: > +1 (binding) > > On Tue, Jan 23, 2018 at 7:28 AM, Matt Farmer wrote: > > > +1 from me (non-binding) =) > > > > > On Jan 22, 2018, at 7:35 PM, Sönke Liebau INVALID> > > wrote: > >

Re: [ANNOUNCE] New committer: Colin McCabe

2018-09-26 Thread Konstantine Karantasis
Well deserved! Congratulations Colin. -Konstantine On Wed, Sep 26, 2018 at 4:57 AM Srinivas Reddy wrote: > Congratulations Colin 👏 > > - > Srinivas > > - Typed on tiny keys. pls ignore typos.{mobile app} > > On Tue 25 Sep, 2018, 16:39 Ismael Juma, wrote: > > > Hi all, > > > > The PMC for Apach

Re: [EXTERNAL] [DISCUSS] KIP-310: Add a Kafka Source Connector to Kafka Connect

2018-09-26 Thread Konstantine Karantasis
Hi Rhys, thanks for the proposal and apologies for the late feedback. Utilizing Connect to mirror Kafka topics is definitely a plausible proposal for a very useful use case. However, I don't think the apache/kafka repository is the right place to host such a Connector. Currently, no full-featured

Re: [VOTE] KIP-310: Add a Kafka Source Connector to Kafka Connect

2018-09-26 Thread Konstantine Karantasis
Hi Rhys, I just replied in the discussion thread for KIP-310 with my concerns. Thanks for submitting this proposal. -Konstantine On Tue, Sep 11, 2018 at 12:53 AM McCaig, Rhys wrote: > Hi All, > > Bumping this again. > Can I get feedback from some binding vote holders on this KIP? I think its

Re: [EXTERNAL] [DISCUSS] KIP-310: Add a Kafka Source Connector to Kafka Connect

2018-09-27 Thread Konstantine Karantasis
nt and common enough, that there > should be support in the project - and MirrorMaker is, as you mention, > showing its age. > > Cheers, > Rhys > > > > > > On Sep 26, 2018, at 10:42 AM, Konstantine Karantasis < > konstant...@confluent.io> wrote: > > >

Incremental Cooperative Rebalancing

2018-10-02 Thread Konstantine Karantasis
Hey everyone, I'd like to bring to your attention a general design document that was just published in Apache Kafka's wiki space: https://cwiki.apache.org/confluence/display/KAFKA/Incremental+Cooperative+Rebalancing%3A+Support+and+Policies It deals with the subject of Rebalancing of groups in Ka

Re: [VOTE] KIP-215: Add topic regex support for Connect sinks

2017-11-03 Thread Konstantine Karantasis
Nice addition! +1 (non-binding) Konstantine On Fri, Nov 3, 2017 at 4:52 PM, Jeff Klukas wrote: > So sorry for skirting the process there. I wasn't aware of the 72 hour > window and I don't see that mentioned in in > https://cwiki.apache.org/confluence/display/KAFKA/Bylaws#Bylaws-Voting > > Sho

Re: [VOTE] KIP-238: Expose Kafka cluster ID in Connect REST API

2017-12-17 Thread Konstantine Karantasis
+1 Nice KIP. Thanks! - Konstantine On Sat, Dec 16, 2017 at 6:19 AM, Gwen Shapira wrote: > +1 (binding). Thanks! > On Fri, Dec 15, 2017 at 10:55 AM Ted Yu wrote: > > > +1 > > > > On Fri, Dec 15, 2017 at 10:49 AM, Ewen Cheslack-Postava < > e...@confluent.io > > > > > wrote: > > > > > Discussion

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-01-22 Thread Konstantine Karantasis
hould not be only implicitly enabled by omitting to set this config. Of course, it follows, that default value == "eager" until this policy is deprecated. Also, let me note that I have amended the examples and have added the Connect specific detail I promised above. Let me know

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-01-22 Thread Konstantine Karantasis
uation explanation? > > > > 3. cooperative cmeans -> means that only Incremental Cooperative Connect > > protocol is enabled (version 1 or higher). > > > > 4. For the compatibility change, I'm wondering whether we could just use > 2 > > connect protocols instead of 3. Because the user knows

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-01-24 Thread Konstantine Karantasis
ance policy, your suggestion makes sense. I'm happy to update the KIP. I'll just wait a bit in case there's an objection to the more reactive approach that I'm missing right now. Thanks for your comments Jason. Glad you are finding this KIP draft a good start. Konstantine &

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-01-24 Thread Konstantine Karantasis
hip could be used along with incremental cooperative rebalancing in the future to improve assignment of resources (e.g. topic-partitions or connectors and tasks). Therefore, even if they are combined for the same group at some point, I don't foresee an immediate need to deprecate one over th

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-01-28 Thread Konstantine Karantasis
e "eager" or "cooperative" protocol. The question is > how > > > that affects cooperative semantics. In other words, is it safe to allow > > > tasks to continue executing when a rebalance begins even if the > > coordinator > > > ultimate decide

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-01-28 Thread Konstantine Karantasis
mael Juma wrote: > Thanks for the KIP Konstantine. Quick question: introducing a new > serialization format (ie flatbuffers) has major implications. Have we > considered json? If so, why did we reject it? > > Ismael > > On Fri, Jan 11, 2019, 3:44 PM Konstantine Karantasis <

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-02-04 Thread Konstantine Karantasis
easier and bring the changes of this new rebalancing protocol to Kafka clients, beginning with Kafka Connect, in a more applicable and less disruptive way. I'll change the schema descriptions by end of day. Looking forward to your next comments! Konstantine On Mon, Jan 28, 2019 at 5:22

Re: [ANNOUNCE] New Committer: Bill Bejeck

2019-02-13 Thread Konstantine Karantasis
Congrats Bill! -Konstantine On Wed, Feb 13, 2019 at 8:42 PM Srinivas Reddy wrote: > Congratulations Bill 🎉🎉 > > Well deserved!! > > - > Srinivas > > - Typed on tiny keys. pls ignore typos.{mobile app} > > On Thu, 14 Feb, 2019, 11:21 Ismael Juma > > Congratulations Bill! > > > > On Wed, Feb 13,

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-03-05 Thread Konstantine Karantasis
initial proposal just in case we recognized a need for it, but it wasn't absolutely necessary and now KIP-415 has been updated to use the existing Kafka protocol types. Thanks! Konstantine > > On Wed, Feb 6, 2019 at 9:58 PM Boyang Chen wrote: > > > Thanks Konstantine for

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-03-06 Thread Konstantine Karantasis
your insightful comments so far! Konstantine On Tue, Mar 5, 2019 at 11:20 AM Konstantine Karantasis < konstant...@confluent.io> wrote: > > Hi Guozhang, > thanks for your detailed comments! > > I thought it'd be a good idea to have some code supporting my replies. The

[VOTE] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-03-06 Thread Konstantine Karantasis
I'd like to open the vote on KIP-415: Incremental Cooperative Rebalancing in Kafka Connect https://cwiki.apache.org/confluence/display/KAFKA/KIP-415%3A+Incremental+Cooperative+Rebalancing+in+Kafka+Connect a proposal that will allow Kafka Connect to scale significantly the number of connectors and

Re: [VOTE] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-03-07 Thread Konstantine Karantasis
en they can > trigger another rebalance based on V0 where everything will revoke the > tasks before sending join group requests. > > > Guozhang > > On Wed, Mar 6, 2019 at 2:28 PM Konstantine Karantasis < > konstant...@confluent.io> wrote: > > > I'd like t

Re: [VOTE] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-03-11 Thread Konstantine Karantasis
onfluence/display/KAFKA/KIP-415%3A+Incremental+Cooperative+Rebalancing+in+Kafka+Connect > > > and it lgtm. > > > > > > I'm +1 on the KIP. > > > > > > > > > Guozhang > > > > > > > > > On Thu, Mar 7, 2019 at 2:35 PM Konstantine Karan

Re: [VOTE] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

2019-03-15 Thread Konstantine Karantasis
hanks for the great KIP Konstantine! > > > > > > > > > > > > +1 (non-binding) > > > > > > > > > > > > Robert > > > > > > > > > > > > On Thu, Mar 7, 2019 at 2:56 PM Guozhang Wang > > > > > wrot

Re: [DISCUSS] KIP-449: Add connector contexts to Connect worker logs

2019-04-12 Thread Konstantine Karantasis
Thanks for the KIP Randall. It might not be obvious right away, but this is a great improvement when running Connect with multiple connectors or when debugging Connect, for instance in integration tests or system tests. KIP looks good to me overall, I have just a few comments below: 1. In the sni

Re: [DISCUSS] KIP-449: Add connector contexts to Connect worker logs

2019-04-15 Thread Konstantine Karantasis
effort. The other advantage of this approach is that it doesn't have to be > configurable: you can control it via your own logging configuration file > (rather than optionally including it in the "worker" scope on some of the > log messages). Thoughts? What would the "%

Re: [DISCUSS] KIP-454: Expansion of the ConnectClusterState interface

2019-04-19 Thread Konstantine Karantasis
Resending an email I sent this Monday but didn't make it to the mailing list: Hi Chris, thanks for the KIP! I have the following initial comments: 1. In its current form, the code snippet gives the impression that this is a new interface or an interface that completely replaces the existin

Re: [DISCUSS] KIP-440: Extend Connect Converter to support headers

2019-04-22 Thread Konstantine Karantasis
Thanks for the KIP Yaroslav! Apologies for the late comment. However, after reading the KIP it's still not very clear to me what happens to the existing HeaderConverter interface and what's the expectation for existing code implementing this interface out there. Looking at the PR I see that the e

Re: [DISCUSS] KIP-454: Expansion of the ConnectClusterState interface

2019-04-23 Thread Konstantine Karantasis
be retrieved from the herder; it's basically the same > as with the current ConnectClusterStateImpl class but with a few method > calls changed here and there. > > Thanks again for your thoughts! > > Cheers, > > Chris > > On Fri, Apr 19, 2019 at 11:24 AM Konstantine K

Re: [DISCUSS] KIP-454: Expansion of the ConnectClusterState interface

2019-04-29 Thread Konstantine Karantasis
Hi Chris, Thanks for considering my suggestion regarding default implementations for the new methods. However, given that these implementations don't seem to have sane defaults and throw UnsupportedOperationException, I think we'll be better without defaults. Seems that a compile time error is pre

Re: [DISCUSS] KIP-440: Extend Connect Converter to support headers

2019-04-29 Thread Konstantine Karantasis
at > > will > > > > yet > > > > > be serialized by the `HeaderConverter`. Alternatively, the common ` > > > > > org.apache.kafka.common.header.Headers` would correspond to the raw > > > > header > > > > > pairs from the und

Re: [DISCUSS] KIP-465: Add Consolidated Connector Endpoint to Connect REST API

2019-05-06 Thread Konstantine Karantasis
This is a nice improvement, both from an operational standpoint as well as for testing purposes, as we scale the number of connectors that a connect cluster can run. LGTM, thanks for the KIP Dan! On Fri, May 3, 2019 at 2:50 PM Alex Liu wrote: > Good question, > > `info` is probably the best na

Re: [VOTE] KIP-465: Add Consolidated Connector Endpoint to Connect REST API

2019-05-06 Thread Konstantine Karantasis
Useful and concise KIP +1 (non-binding) Konstantine On Mon, May 6, 2019 at 10:43 AM Randall Hauch wrote: > Thanks, Dan. As mentioned on the discussion, this is really a nice little > addition that was alway missing from the API. > > +1 (binding) > > Randall > > On Mon, May 6, 2019 at 9:23 AM d

Re: [VOT] KIP-449: Add connector contexts to Connect worker logs (vote thread)

2019-05-06 Thread Konstantine Karantasis
Great improvement for multi-tenancy. Thanks Randall! +1 (non-binding) Konstantine On Tue, Apr 30, 2019 at 9:18 PM Chris Egerton wrote: > +1 (non-binding) > > Really looking forward to this. Thanks, Randall! > > On Tue, Apr 30, 2019, 20:47 Magesh Nandakumar > wrote: > > > This will make connec

Re: [DISCUSS] KIP-458: Connector Client Config Override Policy

2019-05-06 Thread Konstantine Karantasis
Thanks for the KIP Magesh, it's quite useful towards the goals for more general multi-tenancy in Connect. Couple of comments from me too: I think the fact that the default policy is 'null' (no implementation) should be mentioned on the table next to the available implementations. Currently the KI

Re: [VOTE] KIP-437: Custom replacement for MaskField SMT

2019-05-06 Thread Konstantine Karantasis
I think is is a useful improvement proposal. Thanks Valeria! I'm +1 (non-binding) on this KIP Konstantine On Mon, Apr 15, 2019 at 2:04 AM Valeria Vasylieva < valeria.vasyli...@gmail.com> wrote: > Hi all, > > Since there are no more objections/proposals I would like to start the > vote on KIP-43

Re: [VOTE] KIP-454: Expansion of the ConnectClusterState interface

2019-05-06 Thread Konstantine Karantasis
ay/KAFKA/KIP-454%3A+Expansion+of+the+ConnectClusterState+interface > > > > > > The discussion thread can be found at > > > https://www.mail-archive.com/dev@kafka.apache.org/msg96911.html > > > > > > Thanks to Konstantine Karantasis and Magesh Nandakumar for their > > thoughtful > > > feedback! > > > > > > Cheers, > > > > > > Chris > > > > > >

Re: [VOTE] KIP-475: New Metric to Measure Number of Tasks on a Connector

2019-06-06 Thread Konstantine Karantasis
Thanks Cyrus, this is a nice and straightforward addition. I'm +1 too, but I'd like to return with a question here as well regarding whether the unassigned tasks will be taken into account. Especially after KIP-415 we might start seeing this status for specific periods of time. Therefore, I think

Re: Preliminary blog post for the Apache Kafka 2.3.0 release

2019-06-18 Thread Konstantine Karantasis
Thanks Colin. Great initiative! Here's a small correction (between **) for KIP-415 with a small suggestion as well (between _ _): In Kafka Connect, worker tasks are distributed among the available worker nodes. When a connector is reconfigured or a new connector is deployed _as well as when a wor

Re: Preliminary blog post for the Apache Kafka 2.3.0 release

2019-06-21 Thread Konstantine Karantasis
in both formats -- blog and video -- for every > release > > > would be helpful as different people have different preferences. > > > > > > Ron > > > > > > On Tue, Jun 18, 2019 at 8:20 PM Colin McCabe > wrote: > > > > > >>

Re: Stopping All Tasks When a New Connector Added

2019-07-22 Thread Konstantine Karantasis
Liu feel free to share your jira account id on a separate email, so one of the committers can add you to the project. Then you or someone else will be able to assign this ticket to you. I'll review the fix some time this week. Thanks! Konstantine On Mon, Jul 22, 2019 at 5:13 AM Liu Luying wrote

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-14 Thread Konstantine Karantasis
Thanks Arjun for tackling the need to support this very useful feature. One thing I noticed while reading the KIP is that I would have loved to see more info regarding how this proposal depends on the underlying logging APIs and implementations. For instance, my understanding is that slf4j can not

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-14 Thread Konstantine Karantasis
19 at 4:38 PM Konstantine Karantasis < konstant...@confluent.io> wrote: > > Thanks Arjun for tackling the need to support this very useful feature. > > One thing I noticed while reading the KIP is that I would have loved to > see more info regarding how this proposal depends o

Re: [VOTE] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-14 Thread Konstantine Karantasis
Thanks for the KIP Arjun. FYI, I left a few comments on the discussion thread, but mentioning here too since I noticed that the vote started a few hours ago. Konstantine On Wed, Aug 14, 2019 at 12:13 AM Cyrus Vafadari wrote: > I am excited to see this implemented +1 nonbinding > > On Tue, Aug

Re: [DISCUSS] KIP-481: SerDe Improvements for Connect Decimal type in JSON

2019-08-14 Thread Konstantine Karantasis
Thanks Almog for preparing this KIP! I think it will improve usability and troubleshooting with JSON data a lot. The finalized plan seems quite concrete now. I also liked that some implementation specific implications (such as setting the ObjectMapper to deserialize floating point as BigDecimal) a

Re: [VOTE] KIP-481: SerDe Improvements for Connect Decimal type in JSON

2019-08-15 Thread Konstantine Karantasis
Thanks Almog! Nicely designed and concise KIP. +1 non-binding Konstantine On Tue, Aug 6, 2019 at 11:44 PM Almog Gavra wrote: > Hello Everyone, > > After discussions on > > https://lists.apache.org/thread.html/fa665a6dc59f73ca294a00bcbef2eaa3ad00cc69626e91c516fa4fca@%3Cdev.kafka.apache.org%3E >

Re: [DISCUSS] KIP-481: SerDe Improvements for Connect Decimal type in JSON

2019-08-15 Thread Konstantine Karantasis
ased on your suggestion (e.g. s/Consumer/Sink > Converter/g and s/Producer/Source Converter/g). > > Cheers, > Almog > > On Wed, Aug 14, 2019 at 8:13 AM Konstantine Karantasis < > konstant...@confluent.io> wrote: > > > Thanks Almog for preparing this KIP! > > I th

Re: [ANNOUNCE] New committer: Damian Guy

2017-06-12 Thread Konstantine Karantasis
Congrats! On Mon, Jun 12, 2017 at 12:48 PM, Michael Noll wrote: > Congratulations, Damian! > > On Mon, Jun 12, 2017 at 9:07 AM, Molnár Bálint > wrote: > > > Congrats, Damien! > > > > 2017-06-12 8:44 GMT+02:00 Rajini Sivaram : > > > > > Congratulations, Damian! > > > > > > On Sat, Jun 10, 2017 a

Re: [ANNOUNCE] New Kafka PMC member Ismael Juma

2017-07-07 Thread Konstantine Karantasis
Congratulations Ismael! On Fri, Jul 7, 2017 at 8:26 AM Eno Thereska wrote: > Congrats! > > Eno > > On 7 Jul 2017, at 16:13, Kamal C wrote: > > > > Congratulations Ismael ! > > > > On 06-Jul-2017 14:11, "Ismael Juma" wrote: > > > >> Thanks everyone! > >> > >> Ismael > >> > >> On Wed, Jul 5, 20

Re: [ANNOUNCE] New Kafka PMC member Jason Gustafson

2017-07-12 Thread Konstantine Karantasis
Congratulations Jason! On Wed, Jul 12, 2017 at 10:54 AM, James Cheng wrote: > Congrats Jason! > > -James > > > On Jul 11, 2017, at 10:32 PM, Guozhang Wang wrote: > > > > Hi Everyone, > > > > Jason Gustafson has been very active in contributing to the Kafka > community > > since he became a Kafk

Re: Way to check if custom SMT has been added to the classpath or even if it i working.

2017-08-14 Thread Konstantine Karantasis
Hi, connector-plugins endpoint does not list the transformations classes currently. However if you are using the latest Kafka version ( >= 0.11.0) one way to see if your transform is discovered during startup in the given classpath is to notice whether a log message such as the one below is printe

Re: [DISCUSS] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-19 Thread Konstantine Karantasis
d this way, > but since it never returned null, this should be fine. > > re: example usage, I've added some screenshot on how this feature would be > used with jconsole. > > Hope this works! > > Thanks very much, > Arjun > > On Wed, Aug 14, 2019 at 6:42 AM Ko

Re: [VOTE] KIP-495: Dynamically Adjust Log Levels in Connect

2019-08-20 Thread Konstantine Karantasis
Thanks for the updates on the KIP Arjun! +1 by me (non-binding) Konstantine On Wed, Aug 14, 2019 at 6:46 AM Konstantine Karantasis < konstant...@confluent.io> wrote: > > Thanks for the KIP Arjun. > > FYI, I left a few comments on the discussion thread, but mentioning

Re: [DISCUSS] KIP-507: Securing Internal Connect REST Endpoints

2019-08-27 Thread Konstantine Karantasis
Thanks for the KIP Chris! This proposal will fill a gap in Kafka Connect deployments and will strengthen their production readiness in even more diverse environments, in which current workarounds are less practical. I've read the discussions regarding why the rebalancing protocol is used here and

KIP-521: Enable redirection of Connect's log4j messages to a file by default

2019-09-11 Thread Konstantine Karantasis
Hi all. While we are in the midst of some very interesting KIP discussions, I'd like to bring a brief and useful KIP on the table as well. It's about enabling redirection of log4j logging to a file for Kafka Connect by default, in a way similar to how this is done for Kafka brokers today. You mi

[DISCUSS] KIP-521: Enable redirection of Connect's log4j messages to a file by default

2019-09-11 Thread Konstantine Karantasis
*** Missed the [DISCUSS] tag in the previous email. Reposting here, please reply in this thread instead *** Hi all. While we are in the midst of some very interesting KIP discussions, I'd like to bring a brief and useful KIP on the table as well. It's about enabling redirection of log4j logging

Re: [DISCUSS] KIP-521: Enable redirection of Connect's log4j messages to a file by default

2019-09-11 Thread Konstantine Karantasis
wrote: > Great idea. It will greatly improve the ops experience. Can't believe > we didn't do it before. > > On Wed, Sep 11, 2019 at 2:07 PM Konstantine Karantasis > wrote: > > > > *** Missed the [DISCUSS] tag in the previous email. Reposting here, >

Re: [DISCUSS] KIP-521: Enable redirection of Connect's log4j messages to a file by default

2019-09-18 Thread Konstantine Karantasis
nd the console would be preferred, and to have users change it in > one place. WDYT? > > Randall > > On Wed, Sep 11, 2019 at 7:06 PM Konstantine Karantasis < > konstant...@confluent.io> wrote: > > > Thanks Gwen! > > Indeed, it's a common setup and it's been

[VOTE] KIP-521: Redirect Connect log4j messages also to a file by default

2019-09-19 Thread Konstantine Karantasis
I'd like to start the vote on KIP-521. The proposal seems straightforward and no major concerns came up during its recent brief discussions. I'd appreciate your votes and, of course, your comments are still welcome. Hopefully we could meet the forthcoming KIP deadline for this simple yet useful o

Re: [VOTE] KIP-511: Collect and Expose Client's Name and Version in the Brokers

2019-09-20 Thread Konstantine Karantasis
Quite useful KIP from an operational standpoint and I also like it in its most recent revised form. +1 (non-binding). Konstantine On Fri, Sep 20, 2019 at 9:55 AM Gwen Shapira wrote: > +1 (binding) > > Thanks for the KIP, David and for the help with the design, Colin. I > think it looks great

Re: [VOTE] KIP-507: Securing Internal Connect REST Endpoints

2019-09-24 Thread Konstantine Karantasis
Nicely done! +1 (non-binding) Konstantine On Tue, Sep 24, 2019 at 9:10 AM Rajini Sivaram wrote: > Thanks for the KIP, Chris! > > +1 (binding) > > Regards, > > Rajini > > On Tue, Sep 24, 2019 at 3:12 PM Randall Hauch wrote: > > > Thanks, Chris! > > > > +1 (binding) > > > > On Thu, Sep 19, 2019

Re: [VOTE] KIP-521: Redirect Connect log4j messages also to a file by default

2019-09-24 Thread Konstantine Karantasis
, 2019, 4:37 PM Randall Hauch wrote: > > > > > +1 (binding) > > > > > > Great work, Konstantine. > > > > > > On Thu, Sep 19, 2019 at 11:50 AM Gwen Shapira > wrote: > > > > > > > +1 (binding) > > > > > > > > On T

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-09-25 Thread Konstantine Karantasis
Hi Manikumar, let's add one more on the list that was voted on time and is straightforward: KIP-521: https://cwiki.apache.org/confluence/display/KAFKA/KIP-521%3A+Enable+redirection+of+Connect%27s+log4j+messages+to+a+file+by+default with jira ticket: https://issues.apache.org/jira/browse/KAFKA-560

Re: [DISCUSS] Apache Kafka 2.4.0 release

2019-11-13 Thread Konstantine Karantasis
>> >>> > > > > > >>>> > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-444%3A+Augment+metrics+for+Kafka+Streams > > > > > >>>> >>>>> > > > > > >>>> >>>> &g

Re: [ANNOUNCE] New committer: Boyang Chen

2020-06-22 Thread Konstantine Karantasis
Congrats, Boyang! -Konstantine On Mon, Jun 22, 2020 at 9:19 PM Navinder Brar wrote: > Many Congratulations Boyang. Very well deserved. > > Regards,Navinder > > On Tuesday, 23 June, 2020, 07:21:23 am IST, Matt Wang > wrote: > > Congratulations, Boyang! > > > -- > > Best, > Matt Wang > > >

Re: [ANNOUNCE] New committer: Xi Hu

2020-06-28 Thread Konstantine Karantasis
Congrats Xi! -Konstantine On Sat, Jun 27, 2020 at 7:13 PM Matt Wang wrote: > Congratulations ~ > > > -- > > Best, > Matt Wang > > > On 06/26/2020 23:06,David Jacot wrote: > Congrats! > > On Thu, Jun 25, 2020 at 4:08 PM Hu Xi wrote: > > Thank you, everyone. It is my great honor to be a part of

Re: [VOTE] KIP-629: Use racially neutral terms in our codebase

2020-06-28 Thread Konstantine Karantasis
+1 (binding) On Fri, Jun 26, 2020 at 3:10 AM Benny Lee wrote: > > > > From: Tom Bentley > Sent: Friday, 26 June 2020 7:45 PM > To: dev@kafka.apache.org > Subject: Re: [VOTE] KIP-629: Use racially neutral terms in our codebase > > +1 (non-binding) > > On Fri, Ju

Re: [ANNOUNCE] New Kafka PMC Member: Mickael Maison

2020-07-31 Thread Konstantine Karantasis
Congratulations Mickael! -Konstantine On Fri, Jul 31, 2020 at 2:27 PM Ning Zhang wrote: > Congrats Mickael ! > > On 2020/07/31 15:14:30, Jun Rao wrote: > > Hi, Everyone, > > > > Mickael Maison has been a Kafka committer since Nov. 5, 2019. He has > > remained active in the community since beco

Re: [VOTE] KIP-632: Add DirectoryConfigProvider

2020-08-06 Thread Konstantine Karantasis
A bit late on this thread, but I've reviewed the KIP and I'm also +1. Thanks Tom. Happy to see this improvement making it in. Konstantine On Thu, Aug 6, 2020 at 10:27 AM Tom Bentley wrote: > With 3 binding +1 (Manikumar, Mickael and Rajini) and 2 non-binding +1 > (David and William) the vote p

Re: [ANNOUNCE] Apache Kafka 2.6.0

2020-08-06 Thread Konstantine Karantasis
ra Rodoni, Andras Katona, Andrew Olson, Andy Coates, > > > Aneel Nazareth, Anna Povzner, Antony Stubbs, Arjun Satish, Auston, > > avalsa, > > > Badai Aqrandista, belugabehr, Bill Bejeck, Bob Barrett, Boyang Chen, > > Brian > > > Bushree, Brian Byrne, Bruno Cadonna,

Re: [ANNOUNCE] New Kafka PMC Member: John Roesler

2020-08-10 Thread Konstantine Karantasis
Congrats, John! -Konstantine On Mon, Aug 10, 2020 at 4:53 PM Gwen Shapira wrote: > Congratulations, John! > > On Mon, Aug 10, 2020, 1:11 PM Jun Rao wrote: > > > Hi, Everyone, > > > > John Roesler has been a Kafka committer since Nov. 5, 2019. He has > remained > > active in the community since

Re: Please, take a look at my PR

2020-08-26 Thread Konstantine Karantasis
Hi Tess, I've seen the issue and the PR. I'll send a review soon. Thanks for surfacing this issue! Konstantine On Wed, Aug 26, 2020 at 8:19 AM Tess D'erberwill wrote: > Hi, guys! > Please, take a look at my ticket and PR: > https://issues.apache.org/jira/browse/KAFKA-10426 > > This is my fir

Re: [DISCUSS] KIP-580: Exponential Backoff for Kafka Clients

2020-03-25 Thread Konstantine Karantasis
Hi Sanjana and thanks for the KIP! Sorry for the late response, but I still have a few questions that you might find useful. The KIP currently does not mention Kafka Connect at all. I have read the discussion above where it'd been decided to leave Connect and Streams out of the proposed changes,

Re: [VOTE] KIP-580: Exponential Backoff for Kafka Clients

2020-03-25 Thread Konstantine Karantasis
Hi Sanjana. Thanks for the KIP! Seems quite useful not to overwhelm the brokers with the described requests from clients. You have the votes already, and I'm also in favor overall, but I've made a couple of questions (sorry for the delay) regarding Connect, which is also using retry.backoff.ms but

Re: [DISCUSS] KIP-580: Exponential Backoff for Kafka Clients

2020-03-26 Thread Konstantine Karantasis
actually do need both configs, with retry.backkoff.ms < > retry.backoff.max.ms. I will update the KIP to reflect that as well as > incorporate the wording change you have suggested. > > Thanks, > Sanjana > > On Mar 25, 2020, 10:50 AM -0700, Konstantine Karantasis < >

Re: [VOTE] KIP-580: Exponential Backoff for Kafka Clients

2020-03-26 Thread Konstantine Karantasis
n Mar 25, 2020, 10:52 AM -0700, Konstantine Karantasis < > konstant...@confluent.io>, wrote: > > Hi Sanjana. > > Thanks for the KIP! Seems quite useful not to overwhelm the brokers with > > the described requests from clients. > > > > You have the votes already,

Re: [ANNOUNCE] Apache Kafka 2.5.0

2020-04-15 Thread Konstantine Karantasis
dwide, > > > including > > > > > Capital One, Goldman Sachs, ING, LinkedIn, Netflix, Pinterest, > > > Rabobank, > > > > > Target, The New York Times, Uber, Yelp, and Zalando, among others. > > > > > > > > > > A big thank you fo

Re: [VOTE] KIP-131 - Add access to OffsetStorageReader from SourceConnector

2020-05-04 Thread Konstantine Karantasis
I agree with the motivation of this KIP. I think it will enrich the capabilities of source connectors in a quite meaningful way. I'm +1 as well (binding). P.S. Randall I like your idea of enabling connectors to run in old as well as new workers after this KIP. However I think that such connectors

Re: [DISCUSS] KIP-605 Expand Connect Worker Internal Topic Settings

2020-05-04 Thread Konstantine Karantasis
Thanks for the KIP Randall! Automatic creation of internal Connect topics has been very useful since KIP-154 and adapting the Connect workers to allow users to transparently use broker defaults will be useful as well. The KIP looks good. Konstantine On Sun, May 3, 2020 at 10:55 AM Christopher Eg

Re: [VOTE] KIP-605 - Expand Connect Worker Internal Topic Settings

2020-05-11 Thread Konstantine Karantasis
This KIP falls in the category of necessary and straightforward KIPs. Thanks for the nice write-up Randall. +1 (binding) Konstantine On Mon, May 11, 2020 at 8:00 AM Randall Hauch wrote: > Ping for reviewers. > > I guess I never voted, so +1 (binding). > > On Thu, May 7, 2020 at 4:13 PM Christo

Re: [VOTE] KIP-586: Deprecate commit records without record metadata

2020-05-11 Thread Konstantine Karantasis
The proposal makes sense. Thanks for working on this Mario. +1 (binding) Konstantine On Wed, May 6, 2020 at 1:59 PM Randall Hauch wrote: > Thanks for putting this KIP together, Mario. > > +1 (binding) > > Randall > > On Mon, Apr 27, 2020 at 2:05 PM Mario Molina wrote: > > > Hi all, > > > > I'

  1   2   3   4   >