Hello all,
As the number of connect plugins, SMT's, etc have grown, I wanted to bump
this thread to see if there is more interest in adding a Connect REST
endpoint to get the current plugins in the worker.
Thanks all, and thanks Chris for the initial feedback on this.
On Mon, Feb 17, 2020 at 12:56 PM Cyrus Vafadari <cvafad...@gmail.com> wrote:
> Thanks for the feedback, Chris.
>
> WRT Worker-plugins, I see your point that they aren't very useful for
> people trying to create connectors to see what exists already, but I would
> find them useful for things like making assertions against a docker image
> to confirm the pluginpaths/classpaths are configured correctly. It is more
> useful for "sanity check" kinds of things than exploring/browsing. That
> said, I don't feel too strongly and if more feedback from the community
> thinks they are better excluded then we can remove them.
>
> WRT camelCase vs hyphens, I see your point here, will update the KIP.
>
> WRT compatibility -- Good question -- I would expect that yes, new plugins
> would match this format. I will add this explicitly in the KIP for clarity.
> I do think it's a valuable, simple feature to have the worker able to
> report new plugins.
>
> Thanks again!
>
> On Wed, Feb 12, 2020 at 4:44 PM Christopher Egerton <chr...@confluent.io>
> wrote:
>
>> Hi Cyrus,
>>
>> Thanks for the KIP!
>>
>> One quick question--I see the use case for being able to list
>> per-connector
>> plugins (SMTs, converters, and header converters) via REST API, but I'm
>> not
>> so sure about worker plugins (REST extensions and config providers). Since
>> those are configured at startup for the worker, is there any advantage to
>> being able to see the worker plugins available on a running worker? It's
>> not like they can be added during the lifetime of that worker, and that
>> information wouldn't be useful to anyone trying to create a connector as
>> they wouldn't be able to include worker plugins in a connector config
>> anyways.
>>
>> And a small nit--it seems like the precedent set ever-so-slightly by the
>> /connector-plugins resource is to use a hyphen to separate a series of
>> lowercase words in an endpoint path as opposed to camel case with
>> capitalization. What do you think about changing the possible plugin types
>> to "connector", "converter", "header-converter", etc.?
>>
>> One final question about forwards compatibility--the description for the
>> /plugins endpoint indicates that it "Returns plugin descriptions of all
>> types". If another type of pluggable component is added to the framework,
>> should we expect this endpoint to be updated to include that type of
>> plugin
>> as well? And if so, should we also expect a matching /plugins/{pluginType}
>> endpoint to be added?
>>
>> Cheers,
>>
>> Chris
>>
>> On Fri, Sep 6, 2019 at 11:48 AM Cyrus Vafadari <cy...@confluent.io>
>> wrote:
>>
>> > I've updated the KIP here to simplify and clarify the goal -- it's a
>> pretty
>> > simple KIP to add a REST endpoint for SMTs and other plugins. This will
>> > make it easier to see what SMTs you've loaded.
>> >
>> > Hoping for some good discussion!
>> >
>> > Thanks, all
>> >
>> > On Sat, Jul 20, 2019 at 9:07 PM Cyrus Vafadari <cy...@confluent.io>
>> wrote:
>> >
>> > > Hello, all,
>> > >
>> > > I'd like to start discussion on a new KIP I'm proposing to add HTTP
>> > > endpoints to Kafka Connect to give us some more insight into other
>> > > non-connector plugins, like Simple Message Transformations (SMTs). The
>> > KIP
>> > > would not change how Connect works in any meaningful way except to
>> expose
>> > > some more data by the endpoint, analogous to existing
>> > > "ConnectorPluginsResource" class.
>> > >
>> > > Looking forward to getting some feedback.
>> > >
>> > > Cyrus
>> > >
>> > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-494%3A+Connect+REST+Endpoint+for+Transformations+%28SMTs%29+and+other+Plugins
>> > >
>> >
>>
>
