+1 for making this guarantee explicit. It also definitely seems like a good idea to test mixed versions in bigtop.
HDFS is not immune to "new client, old server" scenarios because the HDFS client gets bundled into a lot of places. Colin On Mar 20, 2014 10:55 AM, "Chris Nauroth" <cnaur...@hortonworks.com> wrote: > Our use of protobuf helps mitigate a lot of compatibility concerns, but > there still can be situations that require careful coding on our part. > When adding a new field to a protobuf message, the client might need to do > a null check, even if the server-side implementation in the new version > always populates the field. When adding a whole new RPC endpoint, the > client might need to consider the possibility that the RPC endpoint isn't > there on an old server, and degrade gracefully after the RPC fails. The > original issue in MAPREDUCE-4052 concerned the script commands passed in a > YARN container submission, where protobuf doesn't provide any validation > beyond the fact that they're strings. > > Forward compatibility is harder than backward compatibility, and testing is > a big challenge. Our test suites in the Hadoop repo don't cover this. > Does anyone know if anything in Bigtop tries to run with mixed versions? > > I agree that we need to make it clear in the language that upgrading client > alone is insufficient to get access to new server-side features, including > new YARN APIs. Thanks for the suggestions, Steve. > > Chris Nauroth > Hortonworks > http://hortonworks.com/ > > > > On Thu, Mar 20, 2014 at 5:53 AM, Steve Loughran <ste...@hortonworks.com > >wrote: > > > I'm clearly supportive of this, though of course the testing costs needed > > to back up the assertion make it more expensive than just a statement. > > > > Two issues > > > > -we'd need to make clear that new cluster features that a client can > invoke > > won't be available. You can't expect snapshot or symlink support running > > against a -2.2.0 cluster, even if the client supports it. > > > > -in YARN, there are no guarantees that an app compiled against later YARN > > APIs will work in old clusters. Because YARN apps upload themselves to > the > > server, and run with their hadoop, hdfs & yarn libraries. We have to do a > > bit of introspection in our code already to support this situation. The > > compatibility doc would need to be clear on that too: "YARN apps that use > > new APIs (including new fields in datastructures) can expect link > > exceptions" > > > > > > > > > > > > On 20 March 2014 04:25, Vinayakumar B <vinayakuma...@huawei.com> wrote: > > > > > +1, I agree with your point Chris. It depends on the client application > > > how they using the hdfs jars in their classpath. > > > > > > As implementation already supports the compatibility (through > protobuf), > > > No extra code changes required to support new Client + old server. > > > > > > I feel it will be good to explicitly mention about the compatibility of > > > existing APIs in both versions. > > > > > > Anyway this is not applicable for the new APIs in latest client and > this > > > is understood. We can make it explicit in the document though. > > > > > > > > > Regards, > > > Vinayakumar B > > > > > > -----Original Message----- > > > From: Chris Nauroth [mailto:cnaur...@hortonworks.com] > > > Sent: 20 March 2014 05:36 > > > To: common-dev@hadoop.apache.org > > > Cc: mapreduce-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; > > > yarn-...@hadoop.apache.org > > > Subject: Re: [DISCUSS] Clarification on Compatibility Policy: Upgraded > > > Client + Old Server > > > > > > I think this kind of compatibility issue still could surface for HDFS, > > > particularly for custom applications (i.e. something not executed via > > > "hadoop jar" on a cluster node, where the client classes ought to be > > > injected into the classpath automatically). Running DistCP between 2 > > > clusters of different versions could result in a 2.4.0 client calling a > > > 2.3.0 NameNode. Someone could potentially pick up the 2.4.0 WebHDFS > > > client as a dependency and try to use it to make HTTP calls to a 2.3.0 > > HDFS > > > cluster. > > > > > > Chris Nauroth > > > Hortonworks > > > http://hortonworks.com/ > > > > > > > > > > > > On Wed, Mar 19, 2014 at 4:28 PM, Vinod Kumar Vavilapalli < > > > vino...@apache.org > > > > wrote: > > > > > > > It makes sense only for YARN today where we separated out the > clients. > > > > HDFS is still a monolithic jar so this compatibility issue is kind of > > > > invalid there. > > > > > > > > +vinod > > > > > > > > On Mar 19, 2014, at 1:59 PM, Chris Nauroth <cnaur...@hortonworks.com > > > > > > wrote: > > > > > > > > > I'd like to discuss clarification of part of our compatibility > > policy. > > > > > Here is a link to the compatibility documentation for release > 2.3.0: > > > > > > > > > > > > > > > http://hadoop.apache.org/docs/r2.3.0/hadoop-project-dist/hadoop-common > > > > /Compatibility.html#Wire_compatibility > > > > > > > > > > For convenience, here are the specific lines in question: > > > > > > > > > > Client-Server compatibility is required to allow users to continue > > > > > using the old clients even after upgrading the server (cluster) to > a > > > > > later version (or vice versa). For example, a Hadoop 2.1.0 client > > > > > talking to a Hadoop 2.3.0 cluster. > > > > > > > > > > Client-Server compatibility is also required to allow upgrading > > > > individual > > > > > components without upgrading others. For example, upgrade HDFS from > > > > version > > > > > 2.1.0 to 2.2.0 without upgrading MapReduce. > > > > > > > > > > Server-Server compatibility is required to allow mixed versions > > > > > within an active cluster so the cluster may be upgraded without > > > > > downtime in a > > > > rolling > > > > > fashion. > > > > > > > > > > Notice that there is no specific mention of upgrading the client > > > > > ahead of the server. (There is no clause for "upgraded client + > old > > > > > server".) Based on my experience, this is a valid use case when a > > > > > user wants to > > > > pick > > > > > up a client-side bug fix ahead of the cluster administrator's > > > > > upgrade schedule. > > > > > > > > > > Is it our policy to maintain client compatibility with old clusters > > > > within > > > > > the same major release? I think many of us have assumed that the > > > > > answer > > > > is > > > > > yes and coded our new features accordingly, but it isn't made > > > > > explicit in the documentation. Do we all agree that the answer is > > > > > yes, or is it possibly up for debate depending on the change in > > > > > question? In RFC 2119 lingo, is it a MUST or a SHOULD? Either > way, > > > > > I'd like to update the > > > > policy > > > > > text to make our decision clear. After we have consensus, I can > > > > volunteer > > > > > to file an issue and patch the text of the policy. > > > > > > > > > > This discussion started initially in MAPREDUCE-4052, which involved > > > > > changing our scripting syntax for MapReduce YARN container > > submissions. > > > > We > > > > > settled the question there by gating the syntax change behind a > > > > > configuration option. By default, it will continue using the > > > > > existing syntax currently understood by the pre-2.4.0 NodeManager, > > > > > thus preserving compatibility. We wanted to open the policy > > > > > question for wider > > > > discussion > > > > > though. > > > > > > > > > > Thanks, everyone. > > > > > > > > > > Chris Nauroth > > > > > Hortonworks > > > > > http://hortonworks.com/ > > > > > > > > > > -- > > > > > CONFIDENTIALITY NOTICE > > > > > NOTICE: This message is intended for the use of the individual or > > > > > entity > > > > to > > > > > which it is addressed and may contain information that is > > > > > confidential, privileged and exempt from disclosure under > applicable > > > > > law. If the reader of this message is not the intended recipient, > > > > > you are hereby notified > > > > that > > > > > any printing, copying, dissemination, distribution, disclosure or > > > > > forwarding of this communication is strictly prohibited. If you > have > > > > > received this communication in error, please contact the sender > > > > immediately > > > > > and delete it from your system. Thank You. > > > > > > > > > > > > -- > > > > CONFIDENTIALITY NOTICE > > > > NOTICE: This message is intended for the use of the individual or > > > > entity to which it is addressed and may contain information that is > > > > confidential, privileged and exempt from disclosure under applicable > > > > law. If the reader of this message is not the intended recipient, you > > > > are hereby notified that any printing, copying, dissemination, > > > > distribution, disclosure or forwarding of this communication is > > > > strictly prohibited. If you have received this communication in > error, > > > > please contact the sender immediately and delete it from your system. > > > Thank You. > > > > > > > > > > -- > > > CONFIDENTIALITY NOTICE > > > NOTICE: This message is intended for the use of the individual or > entity > > > to which it is addressed and may contain information that is > > confidential, > > > privileged and exempt from disclosure under applicable law. If the > reader > > > of this message is not the intended recipient, you are hereby notified > > that > > > any printing, copying, dissemination, distribution, disclosure or > > > forwarding of this communication is strictly prohibited. If you have > > > received this communication in error, please contact the sender > > immediately > > > and delete it from your system. Thank You. > > > > > > > -- > > CONFIDENTIALITY NOTICE > > NOTICE: This message is intended for the use of the individual or entity > to > > which it is addressed and may contain information that is confidential, > > privileged and exempt from disclosure under applicable law. If the reader > > of this message is not the intended recipient, you are hereby notified > that > > any printing, copying, dissemination, distribution, disclosure or > > forwarding of this communication is strictly prohibited. If you have > > received this communication in error, please contact the sender > immediately > > and delete it from your system. Thank You. > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
