I always assumed IPC was when it was in memory, feather was when it was persisted as files. That distinction/indirection may be totally moot, though, especially if the rust community is just using IPC directly for file formats.
In my mind, feather V1 came just before IPC and started using IPC since V2. I'm not sure if a feather V3 would ever diverge from IPC format or if feather adds anything that's more filesystem friendly (versus other storage system interfaces) or makes filesystem performance more predictable. Aldrin Montana Computer Science PhD Student UC Santa Cruz On Fri, Jul 22, 2022 at 10:49 AM Andrew Lamb <al...@influxdata.com> wrote: > For what it is worth, in the Rust Arrow community, I have only ever heard > the format referred to as "IPC Format" , we haven't used the term Feather > that I know of. > > On Fri, Jul 22, 2022 at 12:41 PM David Li <lidav...@apache.org> wrote: > > > FeatherV2 is the same thing as the IPC file format ( > > https://arrow.apache.org/faq/). This confusion keeps coming up, so: how > > do we want to brand Arrow IPC? As "Feather", or as "Arrow IPC Files"? It > > seems the latter, but even right now Arrow Dataset still writes files > with > > .feather extension (though there are PRs to update it). I don't think > I've > > seen a formal discussion/decision on whether to keep using Feather or IPC > > as the name (though I likely missed it). > > > > On Fri, Jul 22, 2022, at 12:21, Andrew Lamb wrote: > > > I am not familiar with feather -- there are some Draft PRs to add > support > > > for reading [1] and writing [2] the IPC file format which may be > related > > to > > > your question > > > > > > [1] https://github.com/apache/arrow-datafusion/pull/1858 > > > [2] https://github.com/apache/arrow-datafusion/pull/1893 > > > > > > > > > On Fri, Jul 22, 2022 at 12:10 PM Ian Joiner <iajoiner...@gmail.com> > > wrote: > > > > > >> Hi, > > >> > > >> I'd like to ask about the level of Feather support in Datafusion. Can > we > > >> read and write Feather files? I don't really find any information > while > > >> searching through the repo or checking the docs. > > >> > > >> Thanks, > > >> Ian > > >> > > >
