Re: Update on ARROW-1463, related subtasks and plan for testing and merging

2017-10-13 Thread Siddharth Teotia
Okay, sounds good. On Fri, Oct 13, 2017 at 2:50 PM, Wes McKinney wrote: > It is fine to have not-completely-working states in the refactor > branch. I recommend do whatever is the most expedient thing to help > with making progress. > > - Wes > > On Fri, Oct 13, 2017 at 5:42 PM, Siddharth Teotia

Re: Update on ARROW-1463, related subtasks and plan for testing and merging

2017-10-13 Thread Wes McKinney
It is fine to have not-completely-working states in the refactor branch. I recommend do whatever is the most expedient thing to help with making progress. - Wes On Fri, Oct 13, 2017 at 5:42 PM, Siddharth Teotia wrote: > Li, > > I think there is some confusion. Are you suggesting merging into "ja

Re: Update on ARROW-1463, related subtasks and plan for testing and merging

2017-10-13 Thread Siddharth Teotia
Li, I think there is some confusion. Are you suggesting merging into "java vector refactor" branch or the master? Is it fine to merge stuff on the former branch even though few things are broken (around 10 tests) ? If this is allowed, I can do some cleanup (some documentation, some TODOs suggested

[jira] [Created] (ARROW-1673) [Python] NumPy boolean arrays get converted to uint8 arrays on NdarrayToTensor roundtrip

2017-10-13 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-1673: - Summary: [Python] NumPy boolean arrays get converted to uint8 arrays on NdarrayToTensor roundtrip Key: ARROW-1673 URL: https://issues.apache.org/jira/browse/ARROW-1673

[jira] [Created] (ARROW-1672) [Python] Failure to write Feather bytes column

2017-10-13 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1672: --- Summary: [Python] Failure to write Feather bytes column Key: ARROW-1672 URL: https://issues.apache.org/jira/browse/ARROW-1672 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-1671) [C++] Change arrow::MakeArray to not return Status

2017-10-13 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-1671: --- Summary: [C++] Change arrow::MakeArray to not return Status Key: ARROW-1671 URL: https://issues.apache.org/jira/browse/ARROW-1671 Project: Apache Arrow Issue T

Re: Update on ARROW-1463, related subtasks and plan for testing and merging

2017-10-13 Thread Li Jin
Siddharth, Regarding rename: Yes this can be done later. Tests: I agree having code like https://github.com/apache/ arrow/pull/1164/files#diff-0876c9a0005d1dbaea321ea8d39d79ae is hard to maintain even temporarily. I am not sure what's the best way to resolve test failure wrt removing of the acces

Re: Update on ARROW-1463, related subtasks and plan for testing and merging

2017-10-13 Thread Siddharth Teotia
I am not quite sure of the need to rename the vectors. Why do we need to rename? This would first require us to remove all the vectors generated by FixedValueVectors.java as they are non-nullable scalar vectors. Removing non-nullable vectors is one of the goals, but it can be done once the new infr

Re: Update on ARROW-1463, related subtasks and plan for testing and merging

2017-10-13 Thread Li Jin
Siddharth, Thanks for the update. I think it's fine to move forward with more vectors, but in the mean time, I think we should also prioritize to merge https://github.com/apache/arrow/pull/1164, here are a few comments needs to be addressed. (1) Backward-compatibility: I think there is no way to

Re: Update on ARROW-1463, related subtasks and plan for testing and merging

2017-10-13 Thread Siddharth Teotia
The patch that I have put up https://github.com/apache/arrow/pull/1198 seems to be in a reasonable state. We are now working off a different branch "java vector refactor". Now that we have the basic structure, in order to make quick forward progress, I would like to go ahead and do for other type