Re: list scope question

2017-05-10 Thread Jason Altekruse
I think we opted to not create a user list yet, as using arrow is likely going to require some willingness to poke around in the library until we can fill out the docs and some more complete example usages. So as far as I know this is the place to ask, what were you looking to do? On Wed, May 10,

Re: Say no to zero length batches...

2017-04-17 Thread Jason Altekruse
might decide > to > > > subdivide or concatenate the batches to better communicate the stream. > > > Should an internal system be able to express this concept, sure. Should > > we > > > allow these to be communicated between two separate systems

Re: Say no to zero length batches...

2017-04-14 Thread Jason Altekruse
I'm with Wes on this one. A bunch of systems have constructs that deal with zero length collections, lists, iterators, etc. These are established patterns that everyone knows they need to handle the empty case. Forcing applications to create an unnecessary protocol complexity of a special sentinel

Re: Documentation hosting

2017-01-03 Thread Jason Altekruse
+1 Jason Altekruse Software Engineer at Dremio Apache Arrow Committer On Tue, Jan 3, 2017 at 2:59 PM, Leif Walsh wrote: > +1 this sounds pretty sane > On Fri, Dec 30, 2016 at 06:02 Uwe L. Korn wrote: > > > I just had a look over the Apache Calcite approach and I like it very

Re: Java-C++ integration tests -- on the home stretch

2016-12-09 Thread Jason Altekruse
Congrats guys, great work! On Fri, Dec 9, 2016 at 2:14 PM, Julien Le Dem wrote: > Woot! > 🎉 > > On Fri, Dec 9, 2016 at 2:07 PM, Wes McKinney wrote: > > > We just got the integration test suite (binary compatibility between > > Java and C++) passing in Travis CI today! > > > > https://travis-ci.

Re: [VOTE] Release Apache Arrow 0.1.0 - rc3

2016-10-10 Thread Jason Altekruse
Hello all, Tested: - Java build and unit tests - CPP build and unit tests +1 Jason Altekruse Software Engineer at Dremio Apache Drill Committer On Sun, Oct 9, 2016 at 10:48 AM, Uwe Korn wrote: > Hello, > > I have: > > * build java and run the tests > * build cpp

Re: [VOTE] Release Apache Arrow 0.1.0 - rc1

2016-10-04 Thread Jason Altekruse
cInformation.cmake:36 (include) CMakeLists.txt:22 (project) -- Configuring incomplete, errors occurred! See also "/Users/jaltekruse/Downloads/apache-arrow-0.1.0/python/build/temp.macosx-10.10-x86_64-2.7/CMakeFiles/CMakeOutput.log". error: command 'cmake' failed with exit status 1

Re: Arrow Java Build Failiure

2016-09-01 Thread Jason Altekruse
Are you trying to build using Maven or an IDE? I just checked out the tip of master and was able to build and run the tests with "mvn clean install". Jason Altekruse Software Engineer at Dremio Apache Drill Committer On Thu, Sep 1, 2016 at 11:25 AM, Sanjay Rao wrote: > Hi, >

Re: Code review tools for Arrow patches

2016-05-13 Thread Jason Altekruse
If everyone else would prefer Gerrit, I would be okay with using it exclusively to simplify things. It does have several nice features beyond reviewboard as it manages its own git repository, rather than just patch files. Jason Altekruse Software Engineer at Dremio Apache Drill Committer On Thu

Re: Code review tools for Arrow patches

2016-04-15 Thread Jason Altekruse
It looks like I am going to be a minority opinion here, but I think there is at least a case to make that pull requests area little easier for newcomers. I also have opinions about rebasing branches that are shared publicly or currently under review. While it isn't often a problem, rebasing often i

Re: Code reviews / commit-then-review?

2016-03-03 Thread Jason Altekruse
+1 On Thu, Mar 3, 2016 at 12:58 PM, Jacques Nadeau wrote: > +1. Sounds good to me. > > On Thu, Mar 3, 2016 at 12:35 PM, P. Taylor Goetz > wrote: > > > +1 > > > > I think CTR makes sense at this stage. RTC would slow things down > > considerably. > > > > -Taylor > > > > > On Mar 3, 2016, at 3:00

Re: Getting started guide

2016-02-27 Thread Jason Altekruse
files (using > https://github.com/apache/parquet-cpp) via Arrow data structures > (since pandas requires Arrow to be marshalled to NumPy arrays to be > used). > > - Wes > > On Sat, Feb 27, 2016 at 10:06 AM, Jason Altekruse > wrote: > > The java version of the Arrow projec

Re: Getting started guide

2016-02-27 Thread Jason Altekruse
The java version of the Arrow project is reasonably consumable. The code was extracted from the Apache Drill project which has been using this columnar representation since its inception. Steven Phillips is working on finishing the extraction of the necessary interfaces from Drill over in his fork

Re: Comparing with Parquet

2016-02-25 Thread Jason Altekruse
That being said, sometimes encodings can be complementary to processing. Especially in the case of RLE, if a value is only stored once but stored in a way that it represents a value shared across many rows, you only need to do do the calculation once. This type of optimization is something that I