+1, Great Job. Thanks, Dezhi
On 2022/06/28 08:06:01 Nicolò Boschi wrote: > Hi all, > > I opened a new PIP about Pulsar CLI tools. > Looking forward to seeing comments and suggestions. > > PIP: https://github.com/apache/pulsar/issues/16250 > > I posted a short video that shows how the new tool will work: > https://user-images.githubusercontent.com/23314389/176125261-35e123a1-1826-4553-b912-28d00914c0e4.mp4 > > > > > ## Motivation > > Currently Pulsar comes with a couple of utility scripts with the goal of > managing an existing cluster, test behaviours and verify performances: > these tools are available as SH script inside the `bin` directory. > The `pulsar-admin` is the CLI tool supposed to help users and operators to > configure the system, operate over policies, install functions and much > else. > > This proposal basically aims to solve two different problems: > > 1. `pulsar-admin` is terribly slow. Every time the script is triggered, a > new JVM process is spawned. The JVM process creation is heavy and most of > the time is spent by the JVM initialization process. A very common use case > for cluster operators is to create scripts with several commands with the > goal of initialize the cluster, initialize a specific tenant (namespaces, > topics, policies, functions..); in this case, one JVM is initialized for > each scripts leads to waste of time and resources. > > 2. User experience. The current design of the Pulsar CLIs can be improved. > There are a couple of aspects that may be annoying for a user and can > discourage a user to use Pulsar. > 1. Poking around available commands and options in a CLI tool > (`pulsar-admin` for instance, but it's the same for `pulsar-perf` and > `pulsar-client`) is slow and hard. In order to discover commands and > options you need to use `-h` option and, since the performance issue > pointed at 1., it can be annoying and time-consuming. Autocomplete feature > could be a real game-changer in this context. > 2. Different CLI tools. There are a couple of different shell scripts. > They have different goals and it's okay to keep them separated. However, > they raise a barrier for a non Pulsar expert that doesn't have a convenient > entry-point. > > ## Goal > > Address all the issues in the previous section with a single solution. > > ## API Changes > > A new shell script `bin/pulsar-shell` will be introduced. `bin/pulsar > shell` could be a valid alternative but it's not findable for a newbie user > since no direct file exists. > > ## Implementation > > ### Concepts > The new script `pulsar-shell` will differ from the existing for the > following reasons: > > 1. It's a shell. When you start it, it will wait for commands to be > executed. After the command has been executed, despite its result, the > shell session will not be destroyed and it will wait for another command. > 2. Unifies all the CLI scripts. In `pulsar-shell` you'll be able to run all > the existing CLI commands. This will be done in a way that when a new > command/option is added, the pulsar-shell will be updated accordingly. > 3. It comes with sophisticated autocompletion and command history to highly > improve the UX. > 4. Performance. Since JVM is initiated once, it will gain on performance > thanks to the JVM warmup and internal libraries bootstraps. > 5. It will accept a file or a list of commands (parameter and stdin) to > start a shell, run the commands and close the shell. We'll call it the > `non-interactive` mode and it will ease the cluster operations automations. > > Note that existing tools will not be removed/changed. > > ### Implementation > > The shell implementation will be developed in Java, using a well-known > library called [JLine 3](https://github.com/jline/jline3) for the shell > support. > There will be a new main class that will extends the existing class tools > (e.g. `PulsarAdminTool`) > > #### Configuration > The configuration file taken by default will be `client.conf`, like current > CLI tools. The env setup will be exactly the same as for `pulsar-admin` and > `pulsar-client`. > > #### Autocompletion > JLine3 has great support for autocompletion. The shell java class will > translate current `JCommander` tools to the JLine3 completion API. This > will ensure that all APIs will be up-to-date and covered. > > #### History > JLine3 has built-in support for history, both in-memory and persisted (on > local FS). By default the history will be persisted in the user home > directory. This feature can be turned off for security concerns with a > configuration property. > > #### List of tools > The proposal is to get the following tools: > - bin/pulsar-admin > - bin/pulsar-client > - bin/pulsar-perf > > > ## Rejected Alternatives > > - Create a shell mode for `pulsar-admin`. This won't be a flexible solution > because then we may need the same for other tools. Also it doesn't cover > the CLI unification part. > - Create a brand-new shell mode. Not compatible with current CLI and > another tool to maintain. > - Use another shell library. There are a couple of valid alternatives to > JLine3 but it's wide-spread and the autocompletion API is much more > flexible than others since it's not done with annotations (the shell needs > to translate current JCommander definitions from existing tools) > > > > > Nicolò Boschi >
