> On May 10, 2016, at 8:37 AM, Hitesh Shah <hit...@apache.org> wrote:
>
> There have been various discussions on various JIRAs where upstream projects
> such as YARN apps ( Tez, Slider, etc ) are called out for using the above
> so-called Private APIs. A lot of YARN applications that have been built out
> have picked up various bits and pieces of implementation from MapReduce and
> DistributedShell to get things to work.
>
> A recent example is a backward incompatible change introduced ( where the API
> is not even directly invoked ) in the Credentials class related to the
> ability to read tokens/credentials from a file.
Let’s be careful here. It should be noted that the problem happened
primarily because the application jar appears to have included some hadoop jars
in them. So the API invocation isn’t the problem: it’s the fact that the
implementation under the hood changed. If the application jar didn’t bundle
hadoop jars —especially given that were already on the classpath--this problem
should never have happened.
> This functionality is required by pretty much everyone as YARN provides the
> credentials to the app by writing the credentials/tokens to a local file
> which is read in when UserGroupInformation.getCurrentUser() is invoked.
What you’re effectively arguing is that implementations should never
change for public (and in this case LimitedPrivate) APIs. I don’t think that’s
reasonable. Hadoop is filled with changes in major branches where the
implementations have changed but the internals have been reworked to perform
the work in a slightly different manner.
> This change breaks rolling upgrades for yarn applications from 2.x to 3.x
> (whether we end up supporting rolling upgrades across 2.x to 3.x is a
> separate discussion )
At least today, according to the document attached to YARN-666 (lol),
rolling upgrades are only supported within the same major version.
>
> I would like to change our compatibility docs to state that any API that is
> marked as LimitedPrivate{Mapreduce} implies LimitedPrivate{YARN Applications}.
>
> Comments/concerns?
a) That isn’t good enough. No one reads the compatibility guidelines
as it is given how many times someone says “X” isn’t covered when it clearly is.
b) LimitedPrivate{“YARN Applications”} makes zero sense. At that point
it’s Public and the source should be changed to reflect that. Especially given
those flags impacts things like how the javadocs are generated.
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
