Re: HDFS-13616 : batch listing of multiple directories

2020-03-03 Thread Chao Sun
Hi Steve, Thanks for your valuable feedback and apologies for overlooking the needs from object store and others! I'll look into the PR. > Then, after 3.3.0 is out, someone gets to do the FileSystem implementation, with specification, tests etc. Not me -you are the HDFS team -of course you can do

Re: HDFS-13616 : batch listing of multiple directories

2020-03-02 Thread Steve Loughran
On Sat, 29 Feb 2020 at 00:23, Wei-Chiu Chuang wrote: > Steve, > > You made a great point and I'm sorry this API was implemented without > consideration of other FS implementation. > Thank you for your direct feedback. > > async -- yes > builder -- yes > cancellable -- totally agree > > There are

Re: HDFS-13616 : batch listing of multiple directories

2020-02-28 Thread Wei-Chiu Chuang
Steve, You made a great point and I'm sorry this API was implemented without consideration of other FS implementation. Thank you for your direct feedback. async -- yes builder -- yes cancellable -- totally agree There are good use cases for this API though -- Impala and Presto both require lots