Using a shared filesystem abstract API in Arrow Python libraries [was Re: file-system specification]

2018-07-06 Thread Wes McKinney
hi Martin and Antoine, I apologize I haven't been able to look at this in detail yet. I think this is a valuable initiative; I created a wiki page so we can begin to develop a plan to do the work https://cwiki.apache.org/confluence/display/ARROW/Python+Filesystems+and+Filesystem+API I added a JI

Re: file-system specification

2018-05-15 Thread Antoine Pitrou
Hi Martin, On Wed, 9 May 2018 11:28:15 -0400 Martin Durant wrote: > I have sketched out a possible start of a python-wide file-system > specification > https://github.com/martindurant/filesystem_spec > > This came about from my work in some other (remote) file-systems > implementations for py