Re: Does spark support Apache Arrow

2017-06-29 Thread Nirav Patel
e >> >> On May 20, 2016, at 10:16, Todd wrote: >> >> From the official site http://arrow.apache.org/, Apache Arrow is used >> for Columnar In-Memory storage. I have two quick questions: >> 1. Does spark support Apache Arrow? >> 2. When dataframe is cached

Re: Does spark support Apache Arrow

2016-05-19 Thread Hyukjin Kwon
On May 20, 2016, at 10:16, Todd wrote: > > From the official site http://arrow.apache.org/, Apache Arrow is used for > Columnar In-Memory storage. I have two quick questions: > 1. Does spark support Apache Arrow? > 2. When dataframe is cached in memory, the data are saved in columnar

Re: Does spark support Apache Arrow

2016-05-19 Thread Sun Rui
estions: > 1. Does spark support Apache Arrow? > 2. When dataframe is cached in memory, the data are saved in columnar > in-memory style. What is the relationship between this feature and Apache > Arrow,that is, > when the data is in Apache Arrow format,does spark still need the effor

Does spark support Apache Arrow

2016-05-19 Thread Todd
From the official site http://arrow.apache.org/, Apache Arrow is used for Columnar In-Memory storage. I have two quick questions: 1. Does spark support Apache Arrow? 2. When dataframe is cached in memory, the data are saved in columnar in-memory style. What is the relationship between this