Re: Getting started guide

2016-02-26 Thread Vishnu Viswanath
Thanks Leif, I am not trying to incorporate Arrow to any production system. I am just trying to learn this new DS. If you have come across any blogs or if you can tell what should be the starting steps in using Arrow, could you please let me know. -- Thanks and Regards, Vishnu Viswanath, *www.vis

Re: Should Nullable be a nested type?

2016-02-26 Thread Leif Walsh
I meant "We probably don't want std::vector>" On Fri, Feb 26, 2016 at 10:50 PM Leif Walsh wrote: > In the abstract (since I haven't written any code), let me see if I can > make an argument for considering "nullable int" and "int" to both be > worthwhile "primitive" types, as opposed to "Nullabl

Re: Should Nullable be a nested type?

2016-02-26 Thread Leif Walsh
In the abstract (since I haven't written any code), let me see if I can make an argument for considering "nullable int" and "int" to both be worthwhile "primitive" types, as opposed to "Nullable" being a constructed type over the primitive type "int", in the C++ arena. Let's assume Arrow's use cas

Re: Getting started guide

2016-02-26 Thread Leif Walsh
Arrow doesn't seem to be ready for use yet. I think it's an aspirational project. I'd watch for announcements soon but I wouldn't try to incorporate today. On Fri, Feb 26, 2016 at 2:10 PM Slava B wrote: > Agree, also looking for such tutorial > > On Fri, Feb 26, 2016 at 11:05 AM, Vishnu Viswan

Re: Getting started guide

2016-02-26 Thread Slava B
Agree, also looking for such tutorial On Fri, Feb 26, 2016 at 11:05 AM, Vishnu Viswanath < vishnu.viswanat...@gmail.com> wrote: > Hi All, > > I just joined this list, and would like to know if there is any > documentation on how to get started with Apache Arrow. I am interested in > using arrow a

Getting started guide

2016-02-26 Thread Vishnu Viswanath
Hi All, I just joined this list, and would like to know if there is any documentation on how to get started with Apache Arrow. I am interested in using arrow along with Spark or Flink. Thanks and Regards, Vishnu Viswanath, *www.vishnuviswanath.com *

Re: Comparing with Parquet

2016-02-26 Thread Sourav Mazumder
Thanks to all you guys for the inputs/explanations. Seems like it is pretty interesting project if it is complementing parquet for in memory data storage structure for analytical processing as well as random access within the same format. So far the formats which tried to take care of random acces

Re: Should Nullable be a nested type?

2016-02-26 Thread Wes McKinney
To paraphrase a great poet, "mo' templates, 'mo problems". I agree that some theoretical benefits may be reaped in exchange for significantly higher code complexity / likely lower productivity for both developers and users of the library. We would need to see pragmatic argument why the whole librar

Re: Should Nullable be a nested type?

2016-02-26 Thread Daniel Robinson
In C++ at least, I think making Nullable a template (see here: https://github.com/danrobinson/arrow-demo/blob/master/types.h) would make it easier to both define arbitrary classes and to write parsers that take advantage of template specialization. Re: 2 vs 3 code paths: handling the null-skipping