Gosh, has it really been only since february since I last asked the HDFS dev list to stop adding anything to FileSystem/FileContext APIs without
* mentioning this on the hadoop-common list. * specifying what it does in filesystem.md * with a contract test * a new hasPathCapabilities probe. Throwing UnsupportedOperationException only lets people work out if it is unsupported through invocation. Being able to probe for it is better. * ViewFS support. * And, for any new API, one which works well for high-latency object stores: returning Future<Something> and Future<RemoteIterator<Something> when > 1 result is returned This needs to hold even for pulling something up from HDFS. Because if another FS wants to implement it, they need to know what it does, and have tests to verify this. I say this as someone who has tried to document HDFS rename() semantics and gave up. It's really frustrating that every time someone does an FS API change like this in the past (most recently HDFS-13616) I am the one who has to keep sending the reminders out, and then having to try and clean up/. So what now? Options 1. I roll HDFS-15567 back "please be follow process" 2. Someone does a followup patch with specification and contract test, view FS. Add even more to the java 3. We do as per HADOOP-16898 into an MSyncable interface and then FileSystem & HDFS can implement. ViewFS and filterFS still need to pass through. *If nobody is going to volunteer for the specification/test changes, I'm happy for the rollback. It'll remind people about process, * Pre-emptive Warning: No matter what we do for this patch, I will roll back the next change which adds a new API if it's not accompanied by specification and tests. Unhappily yours, Steve
