Thanks Chris, good clarification! --Yongjun
On Fri, Apr 24, 2015 at 12:36 PM, Chris Nauroth <cnaur...@hortonworks.com> wrote: > Metrics/JMX is covered by our compatibility guidelines: > > http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Comp > atibility.html#MetricsJMX > > > Metrics/JMX is similar to our usage of Protocol Buffers/JSON that I > mentioned. It supports backwards-compatible evolution if the change is > done correctly. Adding new fields/beans is compatible. Changing the > names or data types of existing fields/beans is incompatible. Deleting > existing fields/beans is incompatible. > > --Chris Nauroth > > > > > On 4/24/15, 11:19 AM, "Yongjun Zhang" <yzh...@cloudera.com> wrote: > > >Thanks Allen and Chris! > > > >What about adding new entries to jmx report? Somehow I had the impression > >that if we add new entries to it, it's not considered incompatible. Often > >within the same minor release, we want to add new info to jmx report > >instead of waiting for a major release. > > > >For CLI like fsck, maybe we can add a new command line option to enable > >the > >change, and if the command line option is not enabled, don't change the > >output, so we can still commit the change within the same release line? > > > >Thanks. > > > >--Yongjun > > > > > >On Fri, Apr 24, 2015 at 11:05 AM, Chris Nauroth <cnaur...@hortonworks.com > > > >wrote: > > > >> Allen, thank you for calling this out. I was not aware of this part of > >> the compatibility guidelines. I committed one of those fsck changes in > >> HDFS-7933. I see you flagged the issue as incompatible, which agrees > >>with > >> the compatibility guidelines. > >> > >> "Changing the path of a command, removing or renaming command line > >> options, the order of arguments, or the command return code and output > >> break compatibility and may adversely affect users." > >> > >> Most of this intuitively makes sense. Even ignorant of the > >>compatibility > >> guidelines, I would have known to push back on patches that change the > >> path, remove or rename existing options, or change the order of > >>arguments. > >> > >> HDFS-7933 was an example of an output change, and I find this part of > >>the > >> compatibility guidelines much more challenging. We need to be able to > >> evolve CLI output within a release line. On the protocol side, our use > >>of > >> Protocol Buffers and JSON supports evolution if we use it correctly. > >>How > >> can we achieve the equivalent for the CLI? For example, can we turn > >> HDFS-7933 into a backwards-compatible change if it preserves the old > >> output, and only adds the new information if the user passes a new > >> argument, such as -count-decom? > >> > >> Are there other specific issues that you have in mind for CLI > >> incompatibility problems? Let's see if we can find a way to amend them > >>to > >> satisfy the compatibility guidelines. > >> > >> --Chris Nauroth > >> > >> > >> > >> > >> On 4/24/15, 1:02 AM, "Allen Wittenauer" <a...@altiscale.com> wrote: > >> > >> > > >> >On Apr 24, 2015, at 5:53 AM, Yongjun Zhang <yzh...@cloudera.com> > wrote: > >> > > >> >> > >> >> Basically we are adding two additional lines to the report (as > >> >>highlighted > >> >> above). > >> >> > >> >> Theoretically if a tool parses existing fsck report and expects the > >> >> 'Corrupt blocks" entry to be right after the "Average block > >>replication" > >> >> entry, then the change would fail the tool. But is this really a > >> >>concern? > >> >> > >> >> I guess this is not really a concern, so I don't think this change is > >> >> incompatible. but would anyone please comment? > >> >> > >> > > >> > If it changes the output of a CLI command, it's an incompatible > >> change: > >> > > >> > > >> > >> > http://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Co > >> >mpatibility.html#Command_Line_Interface_CLI > >> > > >> > > >> > Other changes to fsck have been punted to 3.x for the *exact > >>same > >> >reason*. In other cases, committers have violated these rules in > >>branch-2 > >> >(not just to fsck, but to all sorts of other command line bits, even > >> >removing command options!) to the point that our compatibility > >>guarantees > >> >are pretty much useless. It's open season on nuking the ecosystem. :( > >> > > >> > People not following the compat rules is one of the reasons I > >> started > >> >building my own changes and release notes, because we have too many > >> >committers either accidentally committing incompatible changes or just > >> >outright lying about them. (Š and, as much as I hate to say it, the > >>HDFS > >> >project is easily the biggest offender.) > >> > >> > >
