Hi Stephen ,
I am +1 on having both human readable output and json output (with an extra
argument) for CLI commands.
Please make sure, for ALL the CLI commands/sub-commands , json outputs are
present before actually making changes in human readable CLI outputs.
Otherwise, tests would still rely on human readable CLI outputs and it would
require regular maintenance in tests.
Also, whenever there is any modification done in json formats, we need to
ensure backward compatibility of json outputs (i.e, key names should not be
changed).
I would request to create a task JIRA to track these efforts.
Once these tasks are completed, internal tests will be modified accordingly to
parse json outputs only and human readable outputs would not be parsed anymore.
Thanks,
Nilotpal Nandi
On 2022/04/21 11:23:13 Stephen O'Donnell wrote:
> Thanks for the comments Pifta.
>
> I guess my suggestion "for JSON the existing field names and structures
> should remain the same", was a little vague. I agree that with JSON, having:
>
> {
> f1: v1
> f2: v2
> }
>
> Is semantically the same whether f1 or f2 comes first. The point I was
> trying to make is that we should not move the nesting of values, eg going
> to something like this, would not be compatible:
>
> {
> wrapper: {
> f1: v1
> f2: v2
> }
> }
>
> I also agree we should document all compatibility areas - and also define
> what happens to something which is deprecated. Eg should it be removed
> after some number of releases, or remain forever? However I feel it is
> better to break down the discussion into small areas (like this one on just
> the CLI command), as otherwise the discussion could go off in many
> directions for some time. Better we reach agreement on small pieces and
> build up the full picture I think.
>
> On Thu, Apr 21, 2022 at 12:13 PM István Fajth <fapi...@gmail.com> wrote:
>
> > Hi Stephen,
> >
> > thank you for bringing this up, I strongly agree with you, we should get a
> > document about compatibility for sure.
> > On the CLI part, I also agree with all the 4 points in general, though let
> > me add a few things:
> > For #1. JSON as it is defined at: https://www.json.org/json-en.html does
> > not require the location to be the same within an object to mean the same.
> > ("An object is an unordered set of name/value pairs."), though change of
> > the order can be considered as a different output for arrays. ("An array is
> > an ordered collection of values."). With that I think if we provide a real
> > correct JSON output, then object members order should/may change at any
> > point in time. Also adding new fields in objects should not take care of
> > any ordering. In testing or processing the output, every usages of the CLI
> > JSON output should parse the full JSON and should not rely on any custom
> > parsing or regex matching, unless that matching is not sensitive to
> > ordering of object fields...
> > For #2-4 I think it is clear, and I would not add much more here... To
> > reflect on the suggestion from Attila, I think we should also have tests
> > that are checking the textual output of CLI commands with all the possible
> > options, so that if there is a change for any reason, we would have a
> > change in that test, so that it would be easier to document what and how
> > has been changed in the CLI, if we ever want to add such a separate section
> > into the changelog.
> >
> > I also agree on having documented rules about compatibility and how we
> > maintain it, not just on the CLI level but as a whole thing covering
> > protocol, API, CLI, REST and any other publicly available access points,
> > and for all internal or external communication interfaces that Ozone has.
> > CLI is a good start, but we should not just stop there.
> >
> > Pifta
> >
> > Stephen O'Donnell <sodonn...@cloudera.com.invalid> ezt írta (időpont:
> > 2022.
> > ápr. 20., Sze, 11:00):
> >
> > > Discussing compatibility with some team members, we realised that we
> > don't
> > > have anything written down about compatibility guarantees in general.
> > There
> > > are many areas in compatibility, but for this discussion, I would like to
> > > focus on CLI command output.
> > >
> > > Ozone is still relatively immature, and as such we often find a need to
> > add
> > > to or modify the CLI command output to provide more information to help
> > > users of the system, or as features are added and enhanced.
> > >
> > > There are two ways the CLI commands can produce output:
> > >
> > > 1. Human readable format.
> > > 2. JSON format.
> > >
> > > The default output is "Human Readable", with JSON enabled via the
> > "--json"
> > > flag. Not all commands support JSON at the moment, but most do, and we
> > > should endeavour to ensure they all do, and fill the gaps ASAP. I am also
> > > mainly focused on admin type commands (eg ozone admin container info etc)
> > > rather than "key listing" commands, similar to the "ls" output we know
> > from
> > > filesystems and hadoop. However I believe "ozone sh key list" etc already
> > > provide their output in JSON format by default.
> > >
> > > I would like the community to agree on a set of compatibility principals
> > > for CLI command output, and as a starting point for discussion, I would
> > > like to propose the following:
> > >
> > > 1. We should ensure that information is never removed from command
> > output,
> > > unless it is deprecated and removed gracefully after some number of
> > > releases. For JSON the existing field names and location in the JSON
> > > structure should remain the same.
> > >
> > > 2. Information can be added to command output, and the output is still
> > > considered compatible as all existing information is still present.
> > >
> > > 3. Following from (2), this means the format of the "Human Readable"
> > output
> > > may change if, for example, some new field or information is added on an
> > > existing line, or new lines of output are added.
> > >
> > > 4. Due to (3), the "Human Readable" output is not intended to be consumed
> > > programmatically. We recommend only JSON be consumed programmatically, as
> > > existing structures will remain even if new information is added.
> > >
> > > Please let me know your thoughts on the above proposals, or any other
> > > suggestions in the scope of "CLI Output Compatibility" and I will bring
> > the
> > > suggestions together into something we can publish in the docs when we
> > > reach agreement.
> > >
> > > Thanks,
> > >
> > > Stephen.
> > >
> >
> >
> > --
> > Pifta
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org
For additional commands, e-mail: dev-h...@ozone.apache.org
