hi Paul, This is a great discussion to get started. I will review the patch in some more detail and send feedback
> I've pushed Joe's initial GraphSchema.fbs to this branch on my Arrow fork I'm concerned the way this patch is set up right now is a little bit problematic from an IP lineage standpoint (since this is his/Nvidia's code and not yours). Would it be possible for Joe to create a pull request directly for this instead? We can create a branch somewhere where we can collaborate, too, if that helps. Thanks, Wes On Sat, May 19, 2018 at 11:35 PM, Paul Taylor <ptay...@apache.org> wrote: > At GTC San Jose last month, NVidia's Joe Eaton (cc'd) presented on the > nvGraph <https://developer.nvidia.com/nvgraph> team's goals for > accelerating in-memory graph processing and analytics. A major component of > that is advancing and standardizing a common, efficient representation for > graphs that can support a broad ranges of use-cases, from small to large. > > To that end, I'd like to kick off the discussion about native graph > representations in Arrow. > > Joe's team has prepared a preliminary FlatBuffers schema for efficient > columnar representations of the four most common graph formats. It includes > embedded edge and vertex property tables, and is designed to be compatible > with the existing Arrow column types. My initial thoughts are that we could > add an optional 5th Graph Message type, similar to how Tensor Messages are > presently implemented. > > I've pushed Joe's initial GraphSchema.fbs to this branch on my Arrow fork > <https://github.com/trxcllnt/arrow/blob/78f6b6c6a5b9e4e7bf96f5bbc4dfed7528b1cca7/format/GraphSchema_Triples_Quads.fbs>. > From what I understand, the tables have been expanded into separate > definitions for the sake of comprehension, and the final forms will be > collapsed into each distinct Graph type, parameterized by sizes defined at > the top. > > I also understand the nvGraph team supports these layouts natively, > enabling the community to take advantage of high-performance GPU kernels > very early on, and possibly align with libraries like Hornet > <https://github.com/hornet-gt/hornetsnest> (previously cuStinger). > > Cheers, > Paul
