Hi Greg,
I have started a vote thread and added it to the doc. Thanks for all your
help on this. Looking forward to this implementation.
Regards
Snehashis
On Wed, Sep 25, 2024 at 10:38 PM Greg Harris
wrote:
> Hey Snehashis,
>
> I updated the KIP to remove some stale mentions of the soft versi
Hey Snehashis,
I updated the KIP to remove some stale mentions of the soft version
requirements, and the crashing workers on startup. I also added more detail
to the REST API.
IMHO we're ready to move to voting, so please open the thread if you also
believe it is ready.
Thanks!
Greg
On Fri, Au
Hi Greg,
Thanks for the clarification!
I agree with not breaking compatibility and opting to not fail worker
startup by validating converter versions.
Let's also go with the unclosed variation of the requirements as a hard
requirement i.e. 3.8 instead of [3.8]. I have updated the KIP and added a
Hey Snehashis,
Thanks for your reply!
> Deviating
> it from the spec seems unnecessary if we document it accordingly, however
> It's probably less intuitive and can lead to confusion. I would just keep
> it as is but making it simpler is also fine.
It sounds like you don't have a strong opinion
Hi Greg,
No issues, I have been caught up in a few things myself.
I have added the points we discussed. In addition, I have added config
providers as part of the set of plugins which will not support versioning,
for the same reason as to why it is not supported in the other plugins that
are initi
Hi Snehashis,
Sorry for the late reply.
> Heterogeneous dependencies in a multi cluster deployment is highly
discouraged
Right, this remains unchanged in this KIP.
> Let's add the version information for both connector and tasks in the
connector status itself
> once we add these two additions t
Hi Greg,
Thanks for taking a look at this, to conclude on the two points above.
1. I'm okay with the status quo of leaving the dependency management of
plugins to systems outside of the connect runtime as it is now. Given that
the dependencies are homogenous across a connect cluster, it should en
Hey Snehashis,
Sorry for the late reply, and thanks for helping close out the discussion.
> Note that if my assumptions are correct then
> this can happen with the existing framework as well, or is there some
> safeguard from this happening?
Currently, if a cluster has a heterogeneous plugin ins
Hi Greg, Chris
Thanks for the in-depth discussion, I have a couple of discussion points
and would like your thoughts on this.
1) One concern I have with the new addition of 'soft' and 'hard' version
requirements is that there could be a mismatch in the plugin version that
two different tasks are
Hi Greg,
First, an apology! I mistakenly assumed that each plugin appeared only once
in the responses from GET /connector-plugins?connectorsOnly=false. Thank
you for correcting me and pointing out that all versions of each plugin
appear in that response, which does indeed satisfy my desire for use
Hey Chris,
Thanks for your thoughts.
> Won't it still only expose the
> latest version for each plugin, instead of the range of versions
available?
Here is a snippet of the current output of the GET
/connector-plugins?connectorsOnly=false endpoint, after I installed two
versions of the debezium
Hi Greg,
I'm confused about the behavior for GET
/connector-plugins?connectorsOnly=false. Won't it still only expose the
latest version for each plugin, instead of the range of versions available?
I'm hoping we can provide a flow where people need at most two REST calls
to discover 1) the complet
Hey Chris,
Thanks for your comments, and I'm glad that it seems like we're
aligning on the vision here.
> An
> alternative could be to change existing behavior to fail fast on any
> invalid default converter configuration instead of just for invalid
> versions
I suppose if this is landing in 4.0
Hi Greg,
Hope you had a nice weekend! Gonna try to keep things concise.
Concluded points:
RE version recommenders, I agree it's likely that programmatic UIs will
already be able to handle dynamic configuration definitions, and the detail
about SMTs is a great point. I still anticipate some awkwa
Hey Chris,
Thanks for your quick follow up.
> But this risk is already present with
> existing error cases, and I don't see anything that justifies changing
> existing behavior with an invalid converter class, or diverging from it in
> the case of invalid converter versions.
The justification is
Hi Greg,
Thanks for your responses! Continuations of existing discussions:
Regarding crashing the worker on startup--yes, there is also a risk to
allowing it to join the cluster. But this risk is already present with
existing error cases, and I don't see anything that justifies changing
existing
Hey Chris,
Thanks for your comments. I'm glad you like the motivations, Snehashis
wrote that part!
> the configuration syntax for the most basic use case of
> specifying a single desired version is pretty counterintuitive.
I agree, and the "soft" requirement scheme is something I wasn't
explicit
Hi all,
Thanks Greg for updating the KIP, and thanks Snehashis for starting the
work on this originally.
The motivation section makes a pretty convincing case for this kind of
feature. My thoughts are mostly about specific details:
1) I like the support for version ranges (the example demonstrat
Hi Greg,
That is much appreciated. No complaints on the additional scope, I will
make some time out to work on this once we have approval.
Thanks
Snehashis
On Fri, May 10, 2024 at 9:28 PM Greg Harris
wrote:
> Hey Snehashis,
>
> I'm glad to hear you're still interested in this KIP!
> I'm happy
Hey Snehashis,
I'm glad to hear you're still interested in this KIP!
I'm happy to let you drive this, and I apologize for increasing the
scope of work so drastically. To make up for that, I'll volunteer to
be the primary PR reviewer to help get this done quickly once the KIP
is approved.
Thanks,
Hi Greg,
Thanks for the follow up to my original KIP, I am in favour of the
additions made to expand its scope, the addition of range versions
specifically make a lot of sense.
Apologies if I have not publicly worked on this KIP for a long time. The
original work was done when the move to service
Hi all,
I'd like to reboot the discussion on KIP-891:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-891%3A+Running+multiple+versions+of+Connector+plugins
I've made some changes, most notably:
1. Specifying versions for all plugins in Connector configs
(converters, header converters, tran
22 matches
Mail list logo
