This proposal looks good to me. I think it will be a helpful addition to the pulsar admin tool. I like that the design avoids an unnecessary dependency on JCommander.
Thanks, Michael On Mon, Aug 22, 2022 at 1:22 PM Enrico Olivelli <eolive...@gmail.com> wrote: > > Yunze, > > > > Il Lun 22 Ago 2022, 20:20 Enrico Olivelli <eolive...@gmail.com> ha scritto: > > > > > > > Il Lun 22 Ago 2022, 17:07 Yunze Xu <y...@streamnative.io.invalid> ha > > scritto: > > > >> I will take a look. But I think we should also add a trivial example > >> (or a test) in the Apache repo > > > > > I will add integration tests for sure. > Currently the PR is only a prototype without tests. > > Enrico > > , e.g. just print some messages for an > >> extended command. > > > > > > Sure. I will update the PIP > > > > And the JavaDocs of these interfaces should be > >> complete and more clear. > >> > > > > > > I will leave that to the implementation PR > > > > Enrico > > > > > >> Thanks, > >> Yunze > >> > >> > >> > >> > >> > 2022年8月22日 18:37，Enrico Olivelli <eolive...@gmail.com> 写道： > >> > > >> > Yunze, > >> > > >> > Il giorno lun 22 ago 2022 alle ore 08:06 Yunze Xu > >> > <y...@streamnative.io.invalid> ha scritto: > >> >> > >> >> The motivation and goal LGTM, but the API changes look very simple and > >> >> hard to use. Do we have to implement all these interfaces for an admin > >> >> extension? If yes, could you show an example in the proposal as a > >> >> guidance? > >> >> > >> >> For example, if I just want to implement a simple command: > >> >> > >> >> ```bash > >> >> ./bin/pulsar-admin kafka create-topic <my-topic> --partitions > >> <num-partitions> > >> >> ``` > >> >> > >> >> How should I implement these interfaces? > >> > > >> > This is a example for the implementation that I am going to do for JMS > >> > > >> https://github.com/datastax/pulsar-jms/pull/53/files#diff-9afaac9c7dc4b3d674e0623cd3d76348b01537c6095e9b5b8e804f59a481cceeR31 > >> > > >> > it is only a mock command at the moment, but it is good to showcase the > >> feature > >> > > >> > Enrico > >> > > >> > > >> >> > >> >> Thanks, > >> >> Yunze > >> >> > >> >> > >> >> > >> >> > >> >>> 2022年8月18日 16:23，Enrico Olivelli <eolive...@gmail.com> 写道： > >> >>> > >> >>> Hello, > >> >>> I have drafted a PIP around this proposal. > >> >>> > >> >>> PIP link: https://github.com/apache/pulsar/issues/17155 > >> >>> > >> >>> I am preparing an official PR, I already have a working prototype. > >> >>> > >> >>> Copy of the contents of the GH issue is attached for discussion here > >> >>> on the Mailing list. > >> >>> > >> >>> Motivation > >> >>> > >> >>> There are many projects that are in the Pulsar ecosystem like Protocol > >> >>> Handlers (Kafka, MQTT, RabbitMQ) and libraries (JMS…) that need > >> >>> additional tools for operating Pulsar following specific conventions > >> >>> adopted by each project and to handle custom domain objects (like JMS > >> >>> queues, Kafka Consumer Groups...). > >> >>> > >> >>> Some examples: > >> >>> > >> >>> Kafka: tools for inspecting internal systems, SchemaRegistry, > >> >>> Transaction Manager, Consumers Groups > >> >>> JMS: tools to handling Subscriptions and Selectors > >> >>> RabbitMQ: tools to handle Pulsar topics and subscription following the > >> >>> convention > >> >>> > >> >>> This is very important as it is hard to follow the conventions of each > >> >>> project using pulsar-admin and the administrator may inadvertently > >> >>> break the system. > >> >>> > >> >>> This feature will enhance the UX of the Pulsar Admin CLI tools for the > >> >>> benefit of the whole ecosystem and users. > >> >>> > >> >>> Goal > >> >>> > >> >>> As we do for many other components in Pulsar, we need a way to enhance > >> >>> the CLI tools, pulsar-admin and pulsar-shell, with additional commands > >> >>> that are specific to the additional features. > >> >>> > >> >>> The proposal here is to add an extension mechanism to the pulsar-admin > >> >>> (and so pulsar-shell) tool. > >> >>> Following the usual conventions for extensions the extension will be > >> >>> bundled in a .nar file that may contain additional third party > >> >>> libraries. > >> >>> > >> >>> The extension will be able to provide new top level commands > >> >>> > >> >>> pulsar-admin my-command-group my-command arguments… > >> >>> > >> >>> The extension will be able to access the PulsarAdmin API provided by > >> >>> the environment. > >> >>> > >> >>> The extension must not depend directly on the JCommander library but > >> >>> we will provide an API to declare the parameters and the other > >> >>> metadata necessary to document and execute the command. > >> >>> This is very important because during the lifecycle of Pulsar the > >> >>> project may decide to upgrade JCommander to an incompatible version or > >> >>> to drop the dependency at all. > >> >>> > >> >>> API Changes > >> >>> > >> >>> We will introduce a new Maven module pulsar-tools-api that contains > >> >>> the public API that can be used by implementations of the custom > >> >>> commands. > >> >>> > >> >>> The implementation will be bundled in a .nar file with a descriptor > >> >>> with the following fields: > >> >>> > >> >>> factoryClass: xxxxx.CommandFactory > >> >>> name: extension-name > >> >>> description: Description... > >> >>> > >> >>> There are the new classes: > >> >>> > >> >>> /** > >> >>> Access to the environment > >> >>> */ > >> >>> public interface CommandExecutionContext { > >> >>> PulsarAdmin getPulsarAdmin(); > >> >>> Properties getConfiguration(); > >> >>> } > >> >>> > >> >>> > >> >>> /** > >> >>> * Custom command implementation > >> >>> */ > >> >>> public interface CustomCommand { > >> >>> String name(); > >> >>> String description(); > >> >>> List<ParameterDescriptor> parameters(); > >> >>> boolean execute(Map<String, Object> parameters, > >> >>> CommandExecutionContext context) throws Exception; > >> >>> } > >> >>> > >> >>> /** > >> >>> * A group of commands. > >> >>> */ > >> >>> public interface CustomCommandGroup { > >> >>> String name(); > >> >>> String description(); > >> >>> List<CustomCommand> commands(CommandExecutionContext context); > >> >>> } > >> >>> > >> >>> /** > >> >>> * Main entry point of the extension > >> >>> */ > >> >>> public interface CustomCommandFactory { > >> >>> > >> >>> /** > >> >>> * Generate the available command groups. > >> >>> */ > >> >>> List<CustomCommandGroup> commandGroups(CommandExecutionContext > >> context); > >> >>> } > >> >>> > >> >>> @Builder > >> >>> @Getter > >> >>> public final class ParameterDescriptor { > >> >>> @Builder.Default > >> >>> private String name = ""; > >> >>> @Builder.Default > >> >>> private String description = ""; > >> >>> private ParameterType type = ParameterType.STRING; > >> >>> private boolean required = false; > >> >>> } > >> > >>
