Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

2024-01-02 Thread Greg Harris
Hi Snehashis, Thank you for the KIP! This is something I've wanted for a long time. I know the discussion has gone cold, are you still interested in pursuing this feature? I'll make time to review the KIP if you are still accepting comments. Thanks, Greg On Tue, Nov 22, 2022 at 12:29 PM Snehash

Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

2022-11-22 Thread Snehashis
Thanks for the points Sagar. > 1) Should we update the GET /connectors endpoint to include the version of > the plugin that is running? It could be useful to figure out the version of > the plugin or I am assuming it gets returned by the expand=info call? I think this is good to have and possible

Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

2022-11-22 Thread Snehashis
Thanks for the explanation Ashwin. This is an interesting notion. This is something which many connectors implicitly do anyway. There are several connectors which have different methods of interpreting the configurations provided. Often the user has some control over how provided configuration sho

Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

2022-11-22 Thread Sagar
Hey Snehashsih, Thanks for the KIP. It looks like a very useful feature. Couple of small-ish points, let me know what you think: 1) Should we update the GET /connectors endpoint to include the version of the plugin that is running? It could be useful to figure out the version of the plugin or I a

Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

2022-11-22 Thread Ashwin
Hi Snehasis, > IIUC (please correct me if I am wrong here), what you highlighted above, is a versioning scheme for a connector config for the same connector (and not different versions of a connector plugin). Sorry for not being more precise in my wording - I meant registering versions of schema

Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

2022-11-21 Thread Snehashis
Thanks for the input Ashwin. > 1. Can you elaborate on the rejected alternatives ? Suppose connector > config is versioned and has a schema. Then a single plugin (whose > dependencies have not changed) can handle multiple config versions for the > same connector class. IIUC (please correct me if

Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

2022-11-21 Thread Ashwin
Hi Snehasis, This is a really useful feature and thanks for initiating this discussion. I had the following questions - 1. Can you elaborate on the rejected alternatives ? Suppose connector config is versioned and has a schema. Then a single plugin (whose dependencies have not changed) can hand

Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

2022-11-21 Thread Snehashis
Hi Mickael. Thanks for your input. Addressing the point you mentioned below. > 1) Can you explain how this would work with the GET > /{pluginName}/config endpoint? How do you specify a version for a > connector? This API returns the set of configurations for a given connector. Since between versi

RE: Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

2022-11-21 Thread Snehashis
Hi Mickael. Thanks for your input. Addressing the point you mentioned below. > 1) Can you explain how this would work with the GET > /{pluginName}/config endpoint? How do you specify a version for a > connector? This API returns the set of configurations for a given connector. Since between versi

Re: [DISCUSS] KIP-891: Running multiple versions of a connector.

2022-11-21 Thread Mickael Maison
Hi, Thanks for the KIP, this is something that could be really useful! 1) Can you explain how this would work with the GET /{pluginName}/config endpoint? How do you specify a version for a connector? 2) Some connectors come bundled with transformations (for example Debezium). How would multiple

[DISCUSS] KIP-891: Running multiple versions of a connector.

2022-11-21 Thread Snehashis
Hi all, I'd like to start a discussion thread on KIP-891: Running multiple versions of a connector. The KIP aims to add the ability for the connect runtime to run multiple versions of a connector. https://cwiki.apache.org/confluence/display/KAFKA/KIP-891%3A+Running+multiple+versions+of+a+connect