Hi Stephen, thank you for bringing this up, I strongly agree with you, we should get a document about compatibility for sure. On the CLI part, I also agree with all the 4 points in general, though let me add a few things: For #1. JSON as it is defined at: https://www.json.org/json-en.html does not require the location to be the same within an object to mean the same. ("An object is an unordered set of name/value pairs."), though change of the order can be considered as a different output for arrays. ("An array is an ordered collection of values."). With that I think if we provide a real correct JSON output, then object members order should/may change at any point in time. Also adding new fields in objects should not take care of any ordering. In testing or processing the output, every usages of the CLI JSON output should parse the full JSON and should not rely on any custom parsing or regex matching, unless that matching is not sensitive to ordering of object fields... For #2-4 I think it is clear, and I would not add much more here... To reflect on the suggestion from Attila, I think we should also have tests that are checking the textual output of CLI commands with all the possible options, so that if there is a change for any reason, we would have a change in that test, so that it would be easier to document what and how has been changed in the CLI, if we ever want to add such a separate section into the changelog.
I also agree on having documented rules about compatibility and how we maintain it, not just on the CLI level but as a whole thing covering protocol, API, CLI, REST and any other publicly available access points, and for all internal or external communication interfaces that Ozone has. CLI is a good start, but we should not just stop there. Pifta Stephen O'Donnell <sodonn...@cloudera.com.invalid> ezt írta (időpont: 2022. ápr. 20., Sze, 11:00): > Discussing compatibility with some team members, we realised that we don't > have anything written down about compatibility guarantees in general. There > are many areas in compatibility, but for this discussion, I would like to > focus on CLI command output. > > Ozone is still relatively immature, and as such we often find a need to add > to or modify the CLI command output to provide more information to help > users of the system, or as features are added and enhanced. > > There are two ways the CLI commands can produce output: > > 1. Human readable format. > 2. JSON format. > > The default output is "Human Readable", with JSON enabled via the "--json" > flag. Not all commands support JSON at the moment, but most do, and we > should endeavour to ensure they all do, and fill the gaps ASAP. I am also > mainly focused on admin type commands (eg ozone admin container info etc) > rather than "key listing" commands, similar to the "ls" output we know from > filesystems and hadoop. However I believe "ozone sh key list" etc already > provide their output in JSON format by default. > > I would like the community to agree on a set of compatibility principals > for CLI command output, and as a starting point for discussion, I would > like to propose the following: > > 1. We should ensure that information is never removed from command output, > unless it is deprecated and removed gracefully after some number of > releases. For JSON the existing field names and location in the JSON > structure should remain the same. > > 2. Information can be added to command output, and the output is still > considered compatible as all existing information is still present. > > 3. Following from (2), this means the format of the "Human Readable" output > may change if, for example, some new field or information is added on an > existing line, or new lines of output are added. > > 4. Due to (3), the "Human Readable" output is not intended to be consumed > programmatically. We recommend only JSON be consumed programmatically, as > existing structures will remain even if new information is added. > > Please let me know your thoughts on the above proposals, or any other > suggestions in the scope of "CLI Output Compatibility" and I will bring the > suggestions together into something we can publish in the docs when we > reach agreement. > > Thanks, > > Stephen. > -- Pifta
