order to become eligible!
Cheers,
Chris Egerton
Hi all,
I've posted "KIP-454: Expansion of the ConnectClusterState interface",
which proposes that we add provide more information about the Connect
cluster to REST extensions.
The KIP can be found at
https://cwiki.apache.org/confluence/display/KAFKA/KIP-454%3A+Expansion+of+the+ConnectClusterStat
Hi Randall,
Thanks for the KIP. Debugging Connect workers definitely becomes harder as
the number of connectors and tasks increases, and your proposal would
simplify the process of sifting through logs and finding relevant
information faster and more accurately.
I have a couple small comments:
F
is
> a great idea and addition to the interface. Let me know what you think.
>
> Thanks,
> Magesh
>
> On Sat, Apr 13, 2019 at 9:00 PM Chris Egerton wrote:
>
> > Hi all,
> >
> > I've posted "KIP-454: Expansion of the ConnectClusterState interface&qu
this
> information. I think a paragraph describing what are the assumptions with
> respect to what consists a "current" task configuration and how this is
> retrieved would be valuable here.
>
> Best,
> Konstantine
>
>
>
> On Thu, Apr 18, 2019 at 2:45 PM Ch
Hi Magesh,
This is an exciting KIP! I have a few questions/comments but overall I like
the direction it's headed in and hope to see it included in the Connect
framework soon.
1. With the proposed "consumer.", "producer.", and "admin." prefixes, how
will this interact with connectors such as the u
t; are any implementations of an interface outside apache/kafka repo. WDYT?
>
> Konstantine
>
> On Fri, Apr 19, 2019 at 2:26 PM Chris Egerton wrote:
>
> > Hi Konstantine,
> >
> > Thanks for your comments. I'll respond to them in the order you provided
> > the
might require well more than
> just
> > the list of configs but might also want some restriction on values. If
> the
> > concern is about users wanting principal and also other configs, it would
> > still be possible by means of a custom implementation. As is, I would
> >
ss citizen of the
> Herder interface. Let me know what you think.
>
> Thanks
> Magesh
>
> On Thu, Apr 18, 2019 at 2:45 PM Chris Egerton wrote:
>
> > Hi Magesh,
> >
> > Thanks for your comments. I'll address them in the order you provided
> them:
> &g
e `producer.override`, '`consumer.override` and `admin.override` -
> provides better clarity that these are overrides.
>
> I don't have a strong opinion in choosing between #2 and #3. Let me
> know what you think.
>
> Thanks
> Magesh
>
> On Wed, Apr 24, 2019 at
nnectClusterDetails which can include things like groupid, underlying
> kafkaclusterId, standalone or distributed, etc. This will help expose any
> cluster related information in the future.
> Let me know if that would work?
>
> Thanks,
> Magesh
>
> On Wed, Apr 24, 2019 at 4:
r 25, 2019 at 4:18 PM Magesh Nandakumar
wrote:
> HI Chrise,
>
> Overall it looks good to me. Just one last comment - I was wondering if
> ConnectClusterDetail should be an interface just like ConnectClusterState.
>
> Thanks,
> Magesh
>
> On Thu, Apr 25, 2019 at 3:54 PM
your inputs.
>
> Thanks,
> Magesh
>
> On Thu, Apr 25, 2019 at 2:18 PM Chris Egerton wrote:
>
> > Hi Magesh,
> >
> > Agreed that we should avoid `dlq.admin`. I also don't have a strong
> opinion
> > between `connector.` and `.override`, but I have a slight
gt; upgrade their implementations.
>
> Otherwise, the KIP LGTM.
>
> Konstantine
>
> On Thu, Apr 25, 2019 at 10:29 PM Magesh Nandakumar
> wrote:
>
> > Thanks a lot, Chris. The KIP looks good to me.
> >
> > On Thu, Apr 25, 2019 at 7:35 PM Chris Eg
verrides are specified? The former seems more in line with
> > the
> > > current behavior, whereas the "disallows" policy seems useful but not
> > > exactly backward compatible. Should we also offer a "Disallow" policy?
> In
> > > fact,
+1 (non-binding)
Really looking forward to this. Thanks, Randall!
On Tue, Apr 30, 2019, 20:47 Magesh Nandakumar wrote:
> This will make connect debugging so much easier. Thanks a lot for driving
> this Randall.
>
> +1 (non-binding)
>
> Thanks,
> Magesh
>
> On Tue, Apr 30, 2019 at 7:19 PM Jeremy
Hi all,
I'd like to start a vote for KIP-454:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-454%3A+Expansion+of+the+ConnectClusterState+interface
The discussion thread can be found at
https://www.mail-archive.com/dev@kafka.apache.org/msg96911.html
Thanks to Konstantine Karantasis and Mag
Hi Magesh,
This looks great! Very excited to see these changes finally coming to
Connect.
+1 (non-binding)
Cheers,
Chris
On Tue, May 7, 2019 at 9:51 AM Magesh Nandakumar
wrote:
> Hi All,
>
> I would like to start a vote on
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-458%3A+Conne
ead can be found here:
https://www.mail-archive.com/dev@kafka.apache.org/msg97458.html. Current
status: +1 binding, +2 non-binding.
Cheers,
Chris
On Tue, Apr 30, 2019 at 12:40 PM Chris Egerton wrote:
> Hi Konstantine,
>
> I've updated the KIP to add default method implem
Hi Daniyar,
I think you may want to tweak your syntax a little:
public static Serde> List(Serde innerSerde) {
return new ListSerde(innerSerde);
}
Does that work?
Cheers,
Chris
On Wed, May 8, 2019 at 10:29 AM Development wrote:
> Hi John,
>
> I updated JIRA and KIP.
>
> I didn’t know abou
> Hi Chris,
>
> Thanks for the KIP, looks good. I have just one question. Can `
> ConnectClusterState#connectorConfig()` return any sensitive configs like
> passwords?
>
> Thanks,
>
> Rajini
>
> On Wed, May 8, 2019 at 1:30 AM Chris Egerton wrote:
>
> &
> > > >
> > > > +1 (binding)
> > > >
> > > > On Thu, May 2, 2019 at 8:16 PM Magesh Nandakumar <
> mage...@confluent.io
> > >
> > > > wrote:
> > > >
> > > > > Thanks a lot for the work on this KIP Chris
Hi Arjun,
This looks great. The changes to public interface are pretty small and
moving the Log4jController class into the clients package seems like the
right way to go. One question I have--it looks like the purpose of this KIP
is to enable dynamic setting of log levels in the Connect framework,
Thanks Arjun! Looks good to me.
On Thu, Aug 1, 2019 at 12:33 PM Arjun Satish wrote:
> Thanks for the feedback, Chris!
>
> Yes, the example is pretty much how Connect will use the new feature.
> Tweaked the section to make this more clear.
>
> Best,
>
> On Fri, Jul 26
Nice stuff, Arjun! +1 (non-binding)
On Tue, Aug 13, 2019 at 1:55 PM Arjun Satish wrote:
> Hey everyone,
>
> I'd like to start a vote for KIP-495 (
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-495%3A+Dynamically+Adjust+Log+Levels+in+Connect
> ).
> This change will make Connect easier
Hi all,
I'd like to start discussion on a KIP to secure the internal "POST
/connectors//tasks" endpoint for the Connect framework. The proposed
changes address a vulnerability in the framework in its current state that
allows malicious users to write arbitrary task configurations for
connectors; i
The KIP page can be found at
https://cwiki.apache.org/confluence/display/KAFKA/KIP-507%3A+Securing+Internal+Connect+REST+Endpoints,
by the way. Apologies for neglecting to include it in my initial email!
On Wed, Aug 14, 2019 at 12:29 PM Chris Egerton wrote:
> Hi all,
>
> I'd
an be used to give
> out session tokens in a secure way. It seems that any attacker could just
> join the group and sign requests with the provided token. Am I missing
> something?
>
> Ryanne
>
> On Wed, Aug 14, 2019, 2:31 PM Chris Egerton wrote:
>
> > T
ination protocol"? I don't see how these are
> related. Why would it matter if the workers sent Kafka messages vs POST
> requests among themselves?
>
> Ryanne
>
> On Thu, Aug 15, 2019 at 3:57 PM Chris Egerton wrote:
>
> > Hi Ryanne,
> >
> > Yes, i
Thanks Ryanne for the feedback so far, hope to hear from some more of you
soon :)
Cheers,
Chris
On Mon, Aug 19, 2019 at 12:02 PM Chris Egerton wrote:
> Hi Ryanne,
>
> The reasoning for this is provided in the KIP: "There would be no clear
> way to achieve consensus among
gt; > Hi Chris,
> > >
> > > Thanks a lot for the KIP. This will certainly be a useful feature. I
> > would
> > > have preferred to use the topic approach as well but I also understand
> > your
> > > point of view about the operational complexi
else over the
course of the next few days or so I'll be opening this for a vote.
Cheers,
Chris
On Wed, Aug 28, 2019 at 12:21 PM Chris Egerton wrote:
> HI all,
>
> Wow, thanks for all the feedback! Happy to see this proposal getting some
> love :)
>
>
> RE Konstan
rd earlier
> ERROR log messages is sufficient.
>
> Fourth, how will an operator of a Connect cluster know whether this
> internal endpoint is protected by authorization via this feature? And how
> can an operator know which Connect workers are preventing a cluster from
> enabling
ey size for the given key
algorithm. More detail can be found in the KIP if anyone's interested.
Cheers,
Chris
On Tue, Sep 10, 2019 at 3:18 PM Chris Egerton wrote:
> Hi Randall,
>
> Thanks so much for your comprehensive feedback! To avoid creating an
> extremely long res
iguous. How will a user
> know that this is occurring, and is this really an unlikely situation
> (e.g., "This scenario is unlikely to occur with any normal usage of
> Connect.")? Seems like users unintentionally misconfiguring some of their
> Connect workers is quite com
4:28 PM Randall Hauch wrote:
> On Mon, Sep 16, 2019 at 3:06 PM Chris Egerton wrote:
>
> > Hi Randall,
> >
> > The new default value for the key size configuration will be "null". I've
> > clarified this in the KIP. This will still preserve backwards
&g
7;ve renamed the proposed configurations to try to take some of
Randall's suggestions into account; the changes can be summarized as
replacing "internal.request" with "inter.worker", and changing "
inter.worker.key.rotation.interval.ms" to "inter.worker.key.ttl.ms&
Hi all,
I'd like to begin voting on KIP-507:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-507%3A+Securing+Internal+Connect+REST+Endpoints
Thanks to Ryanne, Magesh, Konstantine, Greg, and Randall for the fruitful
discussion!
Cheers,
Chris
>
> > On Tue, Sep 24, 2019 at 9:10 AM Rajini Sivaram
> > wrote:
> >
> > > Thanks for the KIP, Chris!
> > >
> > > +1 (binding)
> > >
> > > Regards,
> > >
> > > Rajini
> > >
> > > On Tue, Sep 24, 2
Hi Manikumar,
Would it be possible to add KIP-507 (
https://cwiki.apache.org/confluence/display/KAFKA/KIP-507%3A+Securing+Internal+Connect+REST+Endpoints)
to the 2.4 release, as it has recently been accepted?
Cheers,
Chris
On Tue, Sep 24, 2019 at 9:33 AM Kevin Lu wrote:
> Hi Manikumar,
>
> Pl
> > >
> >>> > > > >> Thanks, Chris. I still think it's strange to have a
> non-policy,
> >>> > since
> >>> > > > >> there's now special behavior for when the policy is not
> >>> specified.
> >>> > &
>>
> > >>>> >>>> On Thu, Oct 3, 2019 at 8:54 AM Manikumar <
> > >>>> manikumar.re...@gmail.com>
> > >>>> >>> wrote:
> > >>>> >>>>
> > >>>> >>>>> Hi all,
Hi Manikumar,
It looks like the header for KIP-440 is accurate ("KIP-440: Extend Connect
Converter to support headers") but the content appears to correspond to
KIP-481 ("SerDe Improvements for Connect Decimal type in JSON") instead.
Could we double-check and make sure that the summary for KIP-440
Hi Hector,
Thanks for the KIP! Really appreciate the tight scope, hoping this will be
easy to review :)
I only have one question: it looks like our existing primitive converters
(string converter + subclasses of NumberConverter) are hardcoded to play
nicely with null values during deserialization
Thanks Hector! LGTM
On Tue, Jul 25, 2023 at 11:47 AM Hector Geraldino (BLOOMBERG/ 919 3RD A) <
hgerald...@bloomberg.net> wrote:
> Thanks Chris for your quick reply.
>
> Your suggestions make sense, I amended the KIP and added a note to the
> class JavaDocs. Also added unit tests to the companion
+1 (binding)
Thanks Hector!
On Wed, Jul 26, 2023 at 3:18 AM Kamal Chandraprakash <
kamal.chandraprak...@gmail.com> wrote:
> +1 (non-binding). Thanks for the KIP!
>
> On Tue, Jul 25, 2023 at 11:12 PM Yash Mayya wrote:
>
> > Hi Hector,
> >
> > Thanks for the KIP!
> >
> > +1 (non-binding)
> >
> >
; >>>>>
> >>>>> ** The Connector API allows building and running reusable producers
> or
> >>>>> consumers that connect Kafka topics to existing applications or data
> >>>>> systems. For example, a connector to a relational dat
Hi Satish,
Would it be possible to include KIP-949 (
https://cwiki.apache.org/confluence/display/KAFKA/KIP-949%3A+Add+flag+to+enable+the+usage+of+topic+separator+in+MM2+DefaultReplicationPolicy)
in the 3.6.0 release? It passed voting yesterday, and is a very small,
low-risk change that we'd like t
also
> > useful for backward compatibility. I added it to the release plan as
> > an exceptional case.
> >
> > ~Satish.
> >
> > On Thu, 3 Aug 2023 at 21:34, Chris Egerton
> wrote:
> > >
> > > Hi Satish,
> > >
> > > Would it b
Hi Jack,
+1 (binding)
Some friendly, non-blocking suggestions:
- IMO it's still worth specifying that the headers will be read-only; this
clarifies the intended API contract both for reviewers of the KIP who
haven't read the GitHub PR yet, and for developers who may leverage this
new method
- Ma
Hi Yash,
Thanks for driving this, and for putting out a well-written KIP. LGTM!
Cheers,
Chris
On Tue, Aug 22, 2023 at 6:13 AM Yash Mayya wrote:
> Hi all,
>
> I'd like to start a discussion thread for this KIP -
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-970%3A+Deprecate+and+rem
atish.
> > > > >
> > > > > On Wed, 16 Aug 2023 at 15:58, Satish Duggana <
> satish.dugg...@gmail.com
> > > >
> > > > > wrote:
> > > > > >
> > > > > > Hi,
> > > > > > Please
+1 (binding), thanks Yash!
On Wed, Aug 30, 2023 at 1:34 PM Andrew Schofield <
andrew_schofield_j...@outlook.com> wrote:
> Thanks for the KIP. Looks good to me.
>
> +1 (non-binding).
>
> Andrew
>
> > On 30 Aug 2023, at 18:07, Hector Geraldino (BLOOMBERG/ 919 3RD A) <
> hgerald...@bloomberg.net> wr
Cheers,
Chris
On Wed, Aug 30, 2023 at 1:01 PM Chris Egerton wrote:
> Hi Satish,
>
> Wanted to let you know that KAFKA-12879 (
> https://issues.apache.org/jira/browse/KAFKA-12879), a breaking change in
> Admin::listOffsets, has been reintroduced into the code base. Since we
> hav
Hi all,
Can't imagine a worse time to publish a new KIP (it's late on a Friday and
we're in the middle of the 3.6.0 release), but I wanted to put forth
KIP-976 for discussion:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-976%3A+Cluster-wide+dynamic+log+adjustment+for+Kafka+Connect
TL;DR:
future.
> > Don't you foresee a future scope type which may require cluster metadata
> ?
> > In that case, isn't it better to forward the requests to the leader in
> the
> > initial implementation ?
> >
> > I would also recommend an additional system
plan, should we also add a test when the scope parameter
> passed is non-null and neither worker nor cluster? In this case the
> behaviour should be similar to the default case.
>
> 4) I had the same question as Yash regarding persistent cluster-wide
> logging level. I think you ha
was to not have the prefix org.apache.kafka.connect in the keys
> considering
> > it is the root namespace. But since logging can be enabled at a root
> level,
> > can we just use initials like (o.a.k.c) which is also a standard
> practice.
> > Let me know what you th
Hi Satish,
I think this qualifies as a blocker. This API has been around for years now
and, while we don't document it as not exposing duplicates*, it has come
with that implicit contract since its inception. More importantly, it has
also never exposed plugins that cannot be used on the worker. Th
Hi all,
Given the regression raised by Greg Harris (
https://issues.apache.org/jira/browse/KAFKA-15473) on the release thread, I
believe we should generate a new candidate with a fix.
I'm -1 (binding) on this RC.
Best,
Chris
On Tue, Sep 19, 2023 at 10:30 AM Christo Lolov
wrote:
> Heya,
>
> I
Congratulations Yash, well-deserved!
On Thu, Sep 21, 2023, 11:28 Bruno Cadonna wrote:
> Hi all,
>
> The PMC of Apache Kafka is pleased to announce a new Kafka committer
> Yash Mayya.
>
> Yash's major contributions are around Connect.
>
> Yash authored the following KIPs:
>
> KIP-793: Allow sink
Congrats Justine!
On Fri, Sep 22, 2023, 20:47 Guozhang Wang
wrote:
> Congratulations!
>
> On Fri, Sep 22, 2023 at 8:44 PM Tzu-Li (Gordon) Tai
> wrote:
> >
> > Congratulations Justine!
> >
> > On Fri, Sep 22, 2023, 19:25 Philip Nee wrote:
> >
> > > Congrats Justine!
> > >
> > > On Fri, Sep 22, 2
Hi Yash,
Thanks for the KIP! Frankly this feels like an oversight in 875 and I'm
glad we're closing that loop ASAP.
Here are my thoughts:
1. (Nit): IMO "start.state", "initial.state", or "target.state" might be
better than just "state" for the field name in the connector creation
request.
2. W
Hi Satish,
Thanks for running this release!
To verify, I:
- Built from source using Java 11 with both:
- - the 3.6.0-rc2 tag on GitHub
- - the kafka-3.6.0-src.tgz artifact from
https://home.apache.org/~satishd/kafka-3.6.0-rc2/
- Checked signatures and checksums
- Ran the quickstart using the kafk
end for it to be
> the sole purpose of this KIP. However, that said, I really like the idea of
> accepting an "offsets" field in the connector creation request since it'd
> reduce the number of user operations required from 3 (create the connector
> in a STOPPED state; al
ectors that fail to generate task
> configurations, connectors that are paused right after being created etc.).
> I did also try out a small prototype before publishing this KIP and things
> do work as expected when creating a connector in the PAUSED / STOPPED state
> by simply writing the
won't really see any errors
> on attempting to parse a JSON file into Java properties).
>
> Thanks,
> Yash
>
> On Thu, Oct 5, 2023 at 1:39 AM Chris Egerton
> wrote:
>
> > Hi Yash,
> >
> > Looking great! Few more thoughts:
> >
> >
> >
Hi all,
I'd like to call for a vote on KIP-976, which augments the existing dynamic
logger adjustment REST API for Kafka Connect to apply changes cluster-wide
instead on a per-worker basis.
The KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-976:+Cluster-wide+dynamic+log+adjustment+for
Hi Yeikel,
Neat question! And thanks for the link to the RestClient code; very helpful.
I don't believe there's a way to configure Kafka Connect to add these
headers to forwarded requests right now. You may be able to do some kind of
out-of-band proxy magic to intercept forwarded requests and ins
Thanks for the KIP, Yash!
+1 (binding)
On Mon, Oct 9, 2023, 01:12 Yash Mayya wrote:
> Hi all,
>
> I'd like to start a vote on KIP-980 which proposes allowing the creation of
> connectors in a stopped (or paused) state.
>
> KIP -
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-980%3A+A
+1, thanks Stanislav!
On Mon, Oct 9, 2023, 14:02 Bill Bejeck wrote:
> +1
>
> Thanks, Stanislav!
>
> -Bill
>
> On Mon, Oct 9, 2023 at 1:59 PM Ismael Juma wrote:
>
> > Thanks for volunteering Stanislav!
> >
> > Ismael
> >
> > On Mon, Oct 9, 2023 at 10:51 AM Stanislav Kozlovski
> > wrote:
> >
> >
gt; > >>
> > > > >> ** Building real-time streaming data pipelines that reliably get
> data
> > > > >> between systems or applications.
> > > > >>
> > > > >> ** Building real-time streaming applications that transfo
Hi all,
Thanks for the votes! I'll cast a final +1 myself and close the vote out.
This KIP passes with the following +1 votes (and no +0 or -1 votes):
• Greg Harris (binding)
• Yash Mayya (binding)
• Federico Valeri
• Mickael Maison (binding)
• hudeqi
• Hector Geraldino
• Chris Egerton (bi
> > > On Tue, Oct 10, 2023 at 3:05 AM Josep Prat >
> > > wrote:
> > >
> > >> Thanks Stanislav!
> > >>
> > >> ———
> > >> Josep Prat
> > >>
> > >> Aiven Deutschland GmbH
> > >
Congrats, Satish!
On Fri, Oct 27, 2023, 11:04 Jun Rao wrote:
> Hi, Everyone,
>
> Satish Duggana has been a Kafka committer since 2022. He has been very
> instrumental to the community since becoming a committer. It's my pleasure
> to announce that Satish is now a member of Kafka PMC.
>
> Congrat
Hi Taras,
Thanks for the KIP! I have some feedback but ultimately I like this
proposal:
1. The "ssl.engine.factory.class" property was originally added for Kafka
brokers in KIP-519 [1]. It'd be nice to link to that KIP (possibly in a
"Background" section?) so that reviewers who don't have that co
t /
> MM2.
> But I guess, a user that implements its own SslEngineFactory` for a
> the broker just prefers to reuse the code as is.
>
> [1].
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-967%3A+Support+custom+SSL+configuration+for+Kafka+Connect+RestServer#KIP967:Supportcus
+1 (binding)
FWIW, I agree that this change should require a KIP. Gating upgrades of
visibility from private or package-private to protected may be cumbersome,
but it comes with the expectation that downgrades in visibility for those
same classes/methods will also be gated behind a KIP, which is I
No objections, I'm +1 ether way.
On Fri, Nov 3, 2023, 18:50 Sophie Blee-Goldman
wrote:
> I am fine with making them public. Of course in that case we should also
> change the corresponding constructors in ConsumerConfig, AdminConfig, and
> StreamsConfig from protected to public as well, to be co
Hi all,
I'd like to open up KIP-1004 for discussion:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1004%3A+Enforce+tasks.max+property+in+Kafka+Connect
As a brief summary: this KIP proposes that the Kafka Connect runtime start
failing connectors that generate a greater number of tasks than
One potential forcing function could be to allow an unconditional revert of
any commit (including testing and non-testing changes) that can be shown to
have introduced test flakiness. This could at least prevent us from
accruing more flaky tests, though it would obviously not be viable for
existing
Hi Hector,
Thanks for taking a look! I think the key difference between the proposed
behavior and the rejected alternative is that the set of tasks that will be
running with the former is still a complete set of tasks, whereas the set
of tasks in the latter is a subset of tasks. Also noteworthy bu
) and
> there may not be a 3.8.0 release before the 4.0.0 release, would we then be
> forced to punt the removal of "tasks.max.enforce" to a future 5.0.0
> release? I don't have any other comments, and the proposed changes make
> sense to me.
>
> Thanks,
> Yash
Hi all,
There's a lot to catch up on here but I wanted to clarify something.
Regarding this comment from Sophie:
> Yet multiple people in this thread so
far have voiced support for "gating merges on the successful completion of
all parts of the build except tests". Just to be totally clear, I re
build is red or has not completed.
>
> Best,
> David
>
> On Sat, Nov 25, 2023 at 5:25 AM Chris Egerton
> wrote:
>
> > Hi all,
> >
> > There's a lot to catch up on here but I wanted to clarify something.
> > Regarding this comment from Sophie:
> &g
Hi Aaron,
Thanks for publishing this KIP after the feedback on your PR. I'm generally
in favor but have a few questions:
1) Is the consumer mentioned in the KIP the one constructed by the
MirrorSourceConnector for polling replicated records from the source
cluster? If yes, are there any other con
Hi Vedarth and Krish,
Thanks for the KIP! I have to admit I'm a little skeptical; hopefully you
can help me understand the need for these additional images.
1) In the motivation section it's stated that "Several other Apache
projects, like Flink, Spark, Solr, have already released Docker Official
Hi John,
Do you have logs demonstrating that the connector was able to start up
successfully? If not, it's possible that instead of the callback being
dropped, it was properly retained but never invoked because the connector
was blocked.
Cheers,
Chris
On Tue, Apr 30, 2024 at 1:57 PM John Hawkin
-- I don't think this list must be exhaustive, as we
> > can
> > >>>>> always do follow up KIP to also apply the handler to other errors
> to
> > >>>>> expand the scope of the handler. The KIP does mention examples, but
> > it
> > &g
a tricky tradeoff what to expose to users and to avoid
> > foot
> > > > >> guns,
> > > > >>>>> but we added similar handlers to Kafka Streams, and have good
> > > > >> experience
> > > > >>>>> with it. He
Hi Mario,
Thanks for the KIP! Looks good overall. One quick thought--it wasn't
immediately obvious to me why we had to touch on InsertField for this. It
looks like the reason is that we use Struct::get [1] to create a clone of
the input value [2], instead of Struct::getWithoutDefault [3].
It seem
t; exceptions: although the idea is brilliant, the user may still make a
> >>> mistake by implementing the wrong method and see a non-expecting
> >> behaviour.
> >>> For example, he may implement handleRetriable() for
> >> RecordTooLargeException
> &g
gt; the null values will be replace by default values. If we replace the "copy"
> logic with a method in the Struct we remove this behavior.
>
> Isn't it?
>
> Mario.
>
> On Wed, May 8, 2024 at 2:14 PM Chris Egerton
> wrote:
>
> > Hi Mario,
> >
>
forms/ValueToKey.java#L106
On Wed, May 8, 2024 at 11:46 AM Chris Egerton wrote:
> Hi Mario,
>
> I think we could have something like `copy` and `copyWithoutDefaults` to
> get around that, but now that you bring up compatibility, I think it's best
> to hold off on this. I'm
On Thu, May 9, 2024 at 3:28 AM Mario Fiore Vitale
wrote:
> Hi Chris,
>
> > Wouldn't ValueToKey [1] be applicable as well, for example?
> Yes, also that one can be affected.
>
> On Wed, May 8, 2024 at 5:59 PM Chris Egerton
> wrote:
>
> > Wait, just one mor
0 AM Mario Fiore Vitale
wrote:
> Hi Chris,
>
> Thanks for the survey. Do you think I need to update the KIP to put all of
> these?
>
> On Thu, May 9, 2024 at 4:14 PM Chris Egerton
> wrote:
>
> > After doing a brief survey of the SMTs that ship with Connect, it
ese new images would add from a
> >> > user's perspective? I'm hesitant to introduce another image, since it
> >> adds
> >> > to the cognitive burden of people who will inevitably have to answer
> the
> >> > question of "What ar
ther alternatives?
>
> Thanks,
> Mario.
>
> On Fri, May 10, 2024 at 3:40 PM Chris Egerton
> wrote:
>
> > Yes, I think we should just do one KIP for all the SMTs. You don't have
> to
> > implement everything all at once or by yourself, but I don't see why we
&
I've finished updating the KIP; @Mario, please let me know what you think.
On Fri, May 10, 2024 at 10:26 AM Chris Egerton wrote:
> I can do it :)
>
> On Fri, May 10, 2024 at 10:02 AM Mario Fiore Vitale
> wrote:
>
>> Yes, I agree. Unfortunately due to the issu
👍 Done
On Fri, May 10, 2024, 10:55 Mario Fiore Vitale wrote:
> Thanks a lot! I have just a minor comment, should we also update the title
> to be more generic since now it's also related to other SMTs?
>
> On Fri, May 10, 2024 at 4:44 PM Chris Egerton
> wrote:
>
> &g
1 - 100 of 788 matches
Mail list logo
