Re: [ANNOUNCE] New Arrow PMC chair: Wes McKinney

2020-10-25 Thread Ji Liu
Congratulations, Wes! David Li 于2020年10月25日周日 上午9:40写道: > Congratulations Wes! > > Best, > David > > On 10/24/20, Li Jin wrote: > > Congrats Wes! > > > > On Sat, Oct 24, 2020 at 10:05 AM Ying Zhou wrote: > > > >> Congratulations Wes! :) > >> > >> Ying > >> > >> > On Oct 23, 2020, at 7:35 PM, J

Re: [DISSCUSS][JAVA] Avoid set reader/writer indices in FieldVector#getFieldBuffers

2020-08-10 Thread Ji Liu
of pattern for getBuffers (create a new method > that doesn't set reader/writer indices and deprecate getFieldBuffers). But > I think that can be handled in a separate PR? > > Anybody else have thoughts? > > -Micah > > On Tue, Aug 4, 2020 at 11:24 PM Ji Liu wrote: > >

Re: [DISSCUSS][JAVA] Avoid set reader/writer indices in FieldVector#getFieldBuffers

2020-08-04 Thread Ji Liu
rote: > > > FWIW, I lack historical context on how these methods evolved, so I'd > > appreciate insight from anyone who has worked on the java codebase for a > > longer period of time. The current situation seems less then ideal. > > > > On Tue, Aug 4, 2020 a

[DISCUSS] How to extended time value range for Timestamp type?

2020-08-04 Thread Ji Liu
with struct(int64, int32) for storage. Besides ExtensionType, are we considering extending our Timestamp for wider range or maybe a new type for cases above? Thanks, Ji Liu

[DISSCUSS][JAVA] Avoid set reader/writer indices in FieldVector#getFieldBuffers

2020-08-04 Thread Ji Liu
ers 3. Introduce a new getIpcBuffers which is unambiguously used for writing record batches (i.e. in VectorUnloader). 4. Update documentation where it makes sense based on all this conversation. More details and discussions can be seen from the PR and hope to get more feedback. Tha

Re: Java Arrow to C++ Arrow and vice versa

2020-07-21 Thread Ji Liu
Hi Chathura, https://lists.apache.org/thread.html/5bf70a6f1a3fa3e543a92b3217e64465a3b761ca307e8114550f9d8b@%3Cdev.arrow.apache.org%3E has the relevant pointers. Thanks, Ji Liu Chathura Widanage 于2020年7月22日周三 上午3:03写道: > Hi all, > > Was there any particular reason for not writing J

Re: [ANNOUNCE] New Arrow committers: Ji Liu and Liya Fan

2020-06-11 Thread Ji Liu
Thanks everyone for the warm welcome! It's a great honor for me to be a committer. Looking forward to contributing more to the community. Thanks, Ji Liu paddy horan 于2020年6月12日周五 上午8:52写道: > Congrats! > > > From: Micah Kornfield > Sen

[jira] [Created] (ARROW-8870) [Java] Make sure Netty Allocator has correct behavior with empty ArrowBuf

2020-05-20 Thread Ji Liu (Jira)
Ji Liu created ARROW-8870: - Summary: [Java] Make sure Netty Allocator has correct behavior with empty ArrowBuf Key: ARROW-8870 URL: https://issues.apache.org/jira/browse/ARROW-8870 Project: Apache Arrow

Re: Contributing to Arrow

2020-04-25 Thread Ji Liu
Hi Karuppayya, Welcome! If you are interested in this issue, feel free to take it. Thanks, Ji Liu -- From:karuppayya Send Time:2020年4月25日(星期六) 14:21 To:emkornfield Cc:dev Subject:Re: Contributing to Arrow Hi Micah, Thanks for

[jira] [Created] (ARROW-8305) [Java] Add ExtensionType support for visitor API

2020-04-01 Thread Ji Liu (Jira)
Ji Liu created ARROW-8305: - Summary: [Java] Add ExtensionType support for visitor API Key: ARROW-8305 URL: https://issues.apache.org/jira/browse/ARROW-8305 Project: Apache Arrow Issue Type: New

[jira] [Created] (ARROW-8171) Consider pre-allocating memory for fix-width vector in Avro adapter iterator

2020-03-20 Thread Ji Liu (Jira)
Ji Liu created ARROW-8171: - Summary: Consider pre-allocating memory for fix-width vector in Avro adapter iterator Key: ARROW-8171 URL: https://issues.apache.org/jira/browse/ARROW-8171 Project: Apache Arrow

Re: [Discuss] [Java] Implement vector diff functionality

2020-03-11 Thread Ji Liu
Hi Micah, Thanks for your feedback, you have opened an issue for Google's Truth[1] and it was assigned to me, I'll try to use it. Thanks, Ji Liu [1] https://issues.apache.org/jira/browse/ARROW-6931 -- From:Micah Kornf

Re: [Java] Port vector validate functionality

2020-03-11 Thread Ji Liu
Hi Wes and Micah, Thanks for your valuable suggestion, I will create sub-tasks under this issue as follow-up works when this one is finished. Thanks, Ji Liu -- From:Micah Kornfield Send Time:2020年3月11日(星期三) 13:42 To:dev Cc:Ji

[Java] Port vector validate functionality

2020-03-06 Thread Ji Liu
]. Thanks, Ji Liu [1] https://github.com/apache/arrow/blob/6600a39ffe149971afd5ad3c78c2b538cdc03cfd/cpp/src/arrow/array/validate.h [2] https://github.com/apache/arrow/blob/6600a39ffe149971afd5ad3c78c2b538cdc03cfd/java/vector/src/main/java/org/apache/arrow/vector/compare/VectorVisitor.java [3] https

[jira] [Created] (ARROW-8020) [Java] Implement vector validate functionality

2020-03-06 Thread Ji Liu (Jira)
Ji Liu created ARROW-8020: - Summary: [Java] Implement vector validate functionality Key: ARROW-8020 URL: https://issues.apache.org/jira/browse/ARROW-8020 Project: Apache Arrow Issue Type: New

[Discuss] [Java] Implement vector diff functionality

2020-03-05 Thread Ji Liu
track this[2]. Thanks, Ji Liu [1] https://github.com/apache/arrow/blob/6600a39ffe149971afd5ad3c78c2b538cdc03cfd/cpp/src/arrow/array/diff.h [2] https://issues.apache.org/jira/browse/ARROW-8019

[jira] [Created] (ARROW-8019) [Java] Implement vector diff functionality

2020-03-05 Thread Ji Liu (Jira)
Ji Liu created ARROW-8019: - Summary: [Java] Implement vector diff functionality Key: ARROW-8019 URL: https://issues.apache.org/jira/browse/ARROW-8019 Project: Apache Arrow Issue Type: New Feature

Re: [ANNOUNCE] New Arrow PMC member: Neal Richardson

2020-03-05 Thread Ji Liu
Congratulations! -- From:Micah Kornfield Send Time:2020年3月5日(星期四) 23:46 To:dev Subject:Re: [ANNOUNCE] New Arrow PMC member: Neal Richardson Congratulations! On Thu, Mar 5, 2020 at 5:57 AM Fan Liya wrote: > Congratulations, Neal

Re: [ANNOUNCE] New Arrow PMC member: Francois Saint-Jacques

2020-03-05 Thread Ji Liu
Congratulations! -- From:Micah Kornfield Send Time:2020年3月5日(星期四) 23:46 To:dev Subject:Re: [ANNOUNCE] New Arrow PMC member: Francois Saint-Jacques Congratulations! On Thu, Mar 5, 2020 at 5:57 AM Fan Liya wrote: > Congratulation

[jira] [Created] (ARROW-7713) [Java] TastLeak was put at the wrong location

2020-01-28 Thread Ji Liu (Jira)
Ji Liu created ARROW-7713: - Summary: [Java] TastLeak was put at the wrong location Key: ARROW-7713 URL: https://issues.apache.org/jira/browse/ARROW-7713 Project: Apache Arrow Issue Type: Bug

Re: [CI] Java build broken on master

2020-01-16 Thread Ji Liu
, I've opened https://issues.apache.org/jira/browse/ARROW-7599 to track. On Thu, Jan 16, 2020 at 10:49 PM Ji Liu wrote: I was fixing, and will open a PR later. Thanks, Ji Liu -- From:Micah Kornfield Send Time:2020

Re: [CI] Java build broken on master

2020-01-16 Thread Ji Liu
I was fixing, and will open a PR later. Thanks, Ji Liu -- From:Micah Kornfield Send Time:2020年1月17日(星期五) 14:48 To:dev Subject:[CI] Java build broken on master This was due to an unexpected conflict between two

[jira] [Created] (ARROW-7546) [Java] Use new implementation to concat vectors values in batch

2020-01-10 Thread Ji Liu (Jira)
Ji Liu created ARROW-7546: - Summary: [Java] Use new implementation to concat vectors values in batch Key: ARROW-7546 URL: https://issues.apache.org/jira/browse/ARROW-7546 Project: Apache Arrow

[DISCUSS][JAVA] Correct the behavior of ListVector isEmpty

2020-01-10 Thread Ji Liu
opened a PR to fix this[1] and not sure what’s the right behavior for null value, should it return [false, false, true, false] or [false, true, true, false] ? Thanks, Ji Liu [1] https://github.com/apache/arrow/pull/6044

[jira] [Created] (ARROW-7539) [Java] FieldVector getFieldBuffers API should not set reader/writer indices

2020-01-09 Thread Ji Liu (Jira)
Ji Liu created ARROW-7539: - Summary: [Java] FieldVector getFieldBuffers API should not set reader/writer indices Key: ARROW-7539 URL: https://issues.apache.org/jira/browse/ARROW-7539 Project: Apache Arrow

Re: Looking to 1.0

2020-01-03 Thread Ji Liu
Hi, Jacques Nadeau I only found two tickets[1][2], and I am interested in these issues. If you don't mind, I would like to take a close watch and try to resolve later. Thanks, Ji Liu [1] https://issues.apache.org/jira/browse/ARROW-7494 [2] https://issues.apache.org/jira/browse/ARROW

[jira] [Created] (ARROW-7490) [Java] Avro converter should convert attributes and props to FieldType metadata

2020-01-01 Thread Ji Liu (Jira)
Ji Liu created ARROW-7490: - Summary: [Java] Avro converter should convert attributes and props to FieldType metadata Key: ARROW-7490 URL: https://issues.apache.org/jira/browse/ARROW-7490 Project: Apache

[jira] [Created] (ARROW-7472) [Java] Fix some incorrect behavior in UnionListWriter

2019-12-24 Thread Ji Liu (Jira)
Ji Liu created ARROW-7472: - Summary: [Java] Fix some incorrect behavior in UnionListWriter Key: ARROW-7472 URL: https://issues.apache.org/jira/browse/ARROW-7472 Project: Apache Arrow Issue Type: Bug

[jira] [Created] (ARROW-7467) [Java] ComplexCopier does incorrect copy for Map nullable info

2019-12-23 Thread Ji Liu (Jira)
Ji Liu created ARROW-7467: - Summary: [Java] ComplexCopier does incorrect copy for Map nullable info Key: ARROW-7467 URL: https://issues.apache.org/jira/browse/ARROW-7467 Project: Apache Arrow Issue

[jira] [Created] (ARROW-7425) [Java] PromotableWriter support writing FixedSizeList type data

2019-12-18 Thread Ji Liu (Jira)
Ji Liu created ARROW-7425: - Summary: [Java] PromotableWriter support writing FixedSizeList type data Key: ARROW-7425 URL: https://issues.apache.org/jira/browse/ARROW-7425 Project: Apache Arrow

[jira] [Created] (ARROW-7406) [Java] NonNullableStructVector#hashCode should pass hasher to child vectors

2019-12-16 Thread Ji Liu (Jira)
Ji Liu created ARROW-7406: - Summary: [Java] NonNullableStructVector#hashCode should pass hasher to child vectors Key: ARROW-7406 URL: https://issues.apache.org/jira/browse/ARROW-7406 Project: Apache Arrow

[jira] [Created] (ARROW-7405) [Java] ListVector isEmpty API is incorrect

2019-12-16 Thread Ji Liu (Jira)
Ji Liu created ARROW-7405: - Summary: [Java] ListVector isEmpty API is incorrect Key: ARROW-7405 URL: https://issues.apache.org/jira/browse/ARROW-7405 Project: Apache Arrow Issue Type: Bug

Re: [Result] [VOTE] Clarifications and forward compatibility changes for Dictionary Encoding (second iteration)

2019-12-01 Thread Ji Liu
Thanks Micah, I'll take the Java side implementation. Thanks, Ji Liu -- From:Micah Kornfield Send Time:2019年12月2日(星期一) 09:25 To:dev Subject:Re: [Result] [VOTE] Clarifications and forward compatibility changes for Dicti

Re: Datasets and Java

2019-11-27 Thread Ji Liu
Hi Francois, Thanks for the proposal and your effort. I made a simple JNI poc before for RecordBatch/VectorSchemaRoot interaction between Java and C++[1][2]. This may help a little. Thanks, Ji Liu [1] https://github.com/tianchen92/jni-poc-java [2] https://github.com/tianchen92/jni-poc-cpp

[jira] [Created] (ARROW-7264) [Java] RangeEqualsVisitor type check is not correct

2019-11-26 Thread Ji Liu (Jira)
Ji Liu created ARROW-7264: - Summary: [Java] RangeEqualsVisitor type check is not correct Key: ARROW-7264 URL: https://issues.apache.org/jira/browse/ARROW-7264 Project: Apache Arrow Issue Type: Bug

Re: Java API for Arrow Compute

2019-11-25 Thread Ji Liu
needed. Thanks, Ji Liu -- From:Micah Kornfield Send Time:2019年11月26日(星期二) 13:02 To:dev Subject:Re: Java API for Arrow Compute +1 to Liya Fan said. Before starting on the endeavor it would be nice to have concrete use-cases and what

Re: [VOTE] Clarifications and forward compatibility changes for Dictionary Encoding (second iteration)

2019-11-25 Thread Ji Liu
To clarify, we have already implemented option #1 ("It is not required that all dictionary batches occur at the beginning") in the previous PR[1]. So hope this proposal will be accepted and I would like to take follow-up works in Java side if possible. Thanks, Ji Liu [1] https://

[jira] [Created] (ARROW-7259) [Java] Support subfield encoder use different hasher

2019-11-25 Thread Ji Liu (Jira)
Ji Liu created ARROW-7259: - Summary: [Java] Support subfield encoder use different hasher Key: ARROW-7259 URL: https://issues.apache.org/jira/browse/ARROW-7259 Project: Apache Arrow Issue Type: New

Re: [VOTE] Clarifications and forward compatibility changes for Dictionary Encoding (second iteration)

2019-11-25 Thread Ji Liu
+1 (non-binding) Thanks Ji Liu -- From:Fan Liya Send Time:2019年11月26日(星期二) 14:01 To:dev ; Micah Kornfield Cc:Wes McKinney Subject:Re: [VOTE] Clarifications and forward compatibility changes for Dictionary Encoding (second

[jira] [Created] (ARROW-7152) [Java] Delete useless class DiffFunction

2019-11-12 Thread Ji Liu (Jira)
Ji Liu created ARROW-7152: - Summary: [Java] Delete useless class DiffFunction Key: ARROW-7152 URL: https://issues.apache.org/jira/browse/ARROW-7152 Project: Apache Arrow Issue Type: Bug

[jira] [Created] (ARROW-7026) [Java] Remove assertions in MessageSerializer/vector/writer/reader

2019-10-30 Thread Ji Liu (Jira)
Ji Liu created ARROW-7026: - Summary: [Java] Remove assertions in MessageSerializer/vector/writer/reader Key: ARROW-7026 URL: https://issues.apache.org/jira/browse/ARROW-7026 Project: Apache Arrow

[jira] [Created] (ARROW-7021) [Java] UnionFixedSizeListWriter decimal type should check writer index

2019-10-29 Thread Ji Liu (Jira)
Ji Liu created ARROW-7021: - Summary: [Java] UnionFixedSizeListWriter decimal type should check writer index Key: ARROW-7021 URL: https://issues.apache.org/jira/browse/ARROW-7021 Project: Apache Arrow

[jira] [Created] (ARROW-6912) [Java] Extract a common base class for avro converter consumers

2019-10-16 Thread Ji Liu (Jira)
Ji Liu created ARROW-6912: - Summary: [Java] Extract a common base class for avro converter consumers Key: ARROW-6912 URL: https://issues.apache.org/jira/browse/ARROW-6912 Project: Apache Arrow

[jira] [Created] (ARROW-6898) [Java] Fix potential memory leak in ArrowWriter and several test classes

2019-10-15 Thread Ji Liu (Jira)
Ji Liu created ARROW-6898: - Summary: [Java] Fix potential memory leak in ArrowWriter and several test classes Key: ARROW-6898 URL: https://issues.apache.org/jira/browse/ARROW-6898 Project: Apache Arrow

[jira] [Created] (ARROW-6889) [Java] ComplexCopier enable FixedSizeList type & fix RangeEualsVisitor StackOverFlow

2019-10-15 Thread Ji Liu (Jira)
Ji Liu created ARROW-6889: - Summary: [Java] ComplexCopier enable FixedSizeList type & fix RangeEualsVisitor StackOverFlow Key: ARROW-6889 URL: https://issues.apache.org/jira/browse/ARROW-6889 Project: Ap

[jira] [Created] (ARROW-6871) [Java] Enhance TransferPair related parameters check and tests

2019-10-13 Thread Ji Liu (Jira)
Ji Liu created ARROW-6871: - Summary: [Java] Enhance TransferPair related parameters check and tests Key: ARROW-6871 URL: https://issues.apache.org/jira/browse/ARROW-6871 Project: Apache Arrow Issue

[jira] [Created] (ARROW-6853) [Java] Support vector and dictionary encoder use different hasher for calculating hashCode

2019-10-11 Thread Ji Liu (Jira)
Ji Liu created ARROW-6853: - Summary: [Java] Support vector and dictionary encoder use different hasher for calculating hashCode Key: ARROW-6853 URL: https://issues.apache.org/jira/browse/ARROW-6853 Project

[jira] [Created] (ARROW-6850) [Java] Jdbc converter support Null type

2019-10-11 Thread Ji Liu (Jira)
Ji Liu created ARROW-6850: - Summary: [Java] Jdbc converter support Null type Key: ARROW-6850 URL: https://issues.apache.org/jira/browse/ARROW-6850 Project: Apache Arrow Issue Type: New Feature

[jira] [Created] (ARROW-6721) [JAVA] Avro adapter benchmark only runs once in JMH

2019-09-27 Thread Ji Liu (Jira)
Ji Liu created ARROW-6721: - Summary: [JAVA] Avro adapter benchmark only runs once in JMH Key: ARROW-6721 URL: https://issues.apache.org/jira/browse/ARROW-6721 Project: Apache Arrow Issue Type: Bug

[jira] [Created] (ARROW-6710) [Java] Add JDBC adapter test to cover cases which contains some null values

2019-09-26 Thread Ji Liu (Jira)
Ji Liu created ARROW-6710: - Summary: [Java] Add JDBC adapter test to cover cases which contains some null values Key: ARROW-6710 URL: https://issues.apache.org/jira/browse/ARROW-6710 Project: Apache Arrow

Re: Timeline for 0.15.0 release

2019-09-26 Thread Ji Liu
Hi Micah, Hmm, unfortunately, I just found a bug in JDBC adapter and open a PR, could this change catch up with 0.15? See https://github.com/apache/arrow/pull/5511 Thanks, Ji Liu -- From:Micah Kornfield Send Time:2019年9月26日(星期四

[jira] [Created] (ARROW-6709) [JAVA] Jdbc adapter currentIndex should increment when value is null

2019-09-26 Thread Ji Liu (Jira)
Ji Liu created ARROW-6709: - Summary: [JAVA] Jdbc adapter currentIndex should increment when value is null Key: ARROW-6709 URL: https://issues.apache.org/jira/browse/ARROW-6709 Project: Apache Arrow

[jira] [Created] (ARROW-6662) [Java] Implement equals/approxEquals API for VectorSchemaRoot

2019-09-22 Thread Ji Liu (Jira)
Ji Liu created ARROW-6662: - Summary: [Java] Implement equals/approxEquals API for VectorSchemaRoot Key: ARROW-6662 URL: https://issues.apache.org/jira/browse/ARROW-6662 Project: Apache Arrow Issue

[jira] [Created] (ARROW-6661) [Java] Implement APIs like slice to enhance VectorSchemaRoot

2019-09-22 Thread Ji Liu (Jira)
Ji Liu created ARROW-6661: - Summary: [Java] Implement APIs like slice to enhance VectorSchemaRoot Key: ARROW-6661 URL: https://issues.apache.org/jira/browse/ARROW-6661 Project: Apache Arrow Issue

[jira] [Created] (ARROW-6601) [Java] Add benchmark for JDBC adapter to avoid potential regression

2019-09-18 Thread Ji Liu (Jira)
Ji Liu created ARROW-6601: - Summary: [Java] Add benchmark for JDBC adapter to avoid potential regression Key: ARROW-6601 URL: https://issues.apache.org/jira/browse/ARROW-6601 Project: Apache Arrow

[jira] [Created] (ARROW-6600) [Java] Implement dictionary-encoded subfields for Union type

2019-09-18 Thread Ji Liu (Jira)
Ji Liu created ARROW-6600: - Summary: [Java] Implement dictionary-encoded subfields for Union type Key: ARROW-6600 URL: https://issues.apache.org/jira/browse/ARROW-6600 Project: Apache Arrow Issue

[jira] [Created] (ARROW-6484) [Java] Enable create indexType for DictionaryEncoding according to dictionary value count

2019-09-07 Thread Ji Liu (Jira)
Ji Liu created ARROW-6484: - Summary: [Java] Enable create indexType for DictionaryEncoding according to dictionary value count Key: ARROW-6484 URL: https://issues.apache.org/jira/browse/ARROW-6484 Project

Re: [RESULT] [VOTE] Alter Arrow binary protocol to address 8-byte Flatbuffer alignment requirements (2nd vote)

2019-09-06 Thread Ji Liu
the next 4 bytes, which are then 0 to signal EOS.ii. Reader just remember the state, so if it reads the continuation token from the beginning, then read all 8 bytes at the end. Thanks, Ji Liu [1] https://github.com/apache/arrow/pull/5229 [2] https://github.com/apache/arrow/pull/5229

Re: [ANNOUNCE] New committers: Ben Kietzman, Kenta Murata, and Neal Richardson

2019-09-05 Thread Ji Liu
Congratulations! Thanks, Ji Liu -- From:Fan Liya Send Time:2019年9月6日(星期五) 09:28 To:dev Subject:Re: [ANNOUNCE] New committers: Ben Kietzman, Kenta Murata, and Neal Richardson Big congratulations to Ben, Kenta and Neal! Best

[jira] [Created] (ARROW-6472) [Java] ValueVector#accept may has potential cast exception

2019-09-05 Thread Ji Liu (Jira)
Ji Liu created ARROW-6472: - Summary: [Java] ValueVector#accept may has potential cast exception Key: ARROW-6472 URL: https://issues.apache.org/jira/browse/ARROW-6472 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-6464) [Java] Refactor FixedSizeListVector#splitAndTransfer with slice API

2019-09-05 Thread Ji Liu (Jira)
Ji Liu created ARROW-6464: - Summary: [Java] Refactor FixedSizeListVector#splitAndTransfer with slice API Key: ARROW-6464 URL: https://issues.apache.org/jira/browse/ARROW-6464 Project: Apache Arrow

[jira] [Created] (ARROW-6460) [Java] Add unit test for large avro data

2019-09-04 Thread Ji Liu (Jira)
Ji Liu created ARROW-6460: - Summary: [Java] Add unit test for large avro data Key: ARROW-6460 URL: https://issues.apache.org/jira/browse/ARROW-6460 Project: Apache Arrow Issue Type: Sub-task

[jira] [Created] (ARROW-6452) [Java] Overrite ValueVector toString() method

2019-09-03 Thread Ji Liu (Jira)
Ji Liu created ARROW-6452: - Summary: [Java] Overrite ValueVector toString() method Key: ARROW-6452 URL: https://issues.apache.org/jira/browse/ARROW-6452 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-6401) [Java] Implement dictionary-encoded subfields for Struct type

2019-08-30 Thread Ji Liu (Jira)
Ji Liu created ARROW-6401: - Summary: [Java] Implement dictionary-encoded subfields for Struct type Key: ARROW-6401 URL: https://issues.apache.org/jira/browse/ARROW-6401 Project: Apache Arrow Issue

Re: [RESULT] [VOTE] Alter Arrow binary protocol to address 8-byte Flatbuffer alignment requirements (2nd vote)

2019-08-29 Thread Ji Liu
Here is the Java implementation https://github.com/apache/arrow/pull/5229 cc @Wes McKinney @emkornfield Thanks, Ji Liu -- From:Ji Liu Send Time:2019年8月28日(星期三) 17:34 To:emkornfield ; dev Cc:Paul Taylor Subject:Re: [RESULT

Re: [RESULT] [VOTE] Alter Arrow binary protocol to address 8-byte Flatbuffer alignment requirements (2nd vote)

2019-08-28 Thread Ji Liu
I could take the Java implementation and will take a close watch on this issue in the next few days. Thanks, Ji Liu -- From:Micah Kornfield Send Time:2019年8月28日(星期三) 17:14 To:dev Cc:Paul Taylor Subject:Re: [RESULT] [VOTE] Alter

[jira] [Created] (ARROW-6356) [Java] Avro adapter implement Enum type and nested Record type

2019-08-26 Thread Ji Liu (Jira)
Ji Liu created ARROW-6356: - Summary: [Java] Avro adapter implement Enum type and nested Record type Key: ARROW-6356 URL: https://issues.apache.org/jira/browse/ARROW-6356 Project: Apache Arrow Issue

[jira] [Created] (ARROW-6311) [Java] Make ApproxEqualsVisitor accept DiffFunction to make it more flexible

2019-08-21 Thread Ji Liu (Jira)
Ji Liu created ARROW-6311: - Summary: [Java] Make ApproxEqualsVisitor accept DiffFunction to make it more flexible Key: ARROW-6311 URL: https://issues.apache.org/jira/browse/ARROW-6311 Project: Apache Arrow

[Discuss] Support read/write interleaved dictionaries and batches in IPC stream

2019-08-21 Thread Ji Liu
r not, we can parse it properly. In the future, I think we should also support write interleaved dictionaries and batches in IPC stream(created an issue to track this[4]), but not quite clear how to implement this. Any opinions about this are appreciated, thanks! Thanks, Ji Liu [1] https://i

[jira] [Created] (ARROW-6308) [Java] Support write interleaved dictionaries and batches in IPC stream

2019-08-21 Thread Ji Liu (Jira)
Ji Liu created ARROW-6308: - Summary: [Java] Support write interleaved dictionaries and batches in IPC stream Key: ARROW-6308 URL: https://issues.apache.org/jira/browse/ARROW-6308 Project: Apache Arrow

Re: Timeline for 0.15.0 release

2019-08-19 Thread Ji Liu
already checked in, other implementations seems not. iii. ARROW-6202: OutOfMemory in JdbcAdapter[4] Caused by trying to load all records in one contiguous batch, fixed by providing iterator API for iteratively reading in ARROW-6219[5]. Thanks, Ji Liu [1] https://github.com/apache/arrow/pull/4960

[jira] [Created] (ARROW-6289) [Java] Add empty() in UnionVector to create instance

2019-08-18 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6289: - Summary: [Java] Add empty() in UnionVector to create instance Key: ARROW-6289 URL: https://issues.apache.org/jira/browse/ARROW-6289 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-6288) [Java] Implement TypeEqualsVisitor comparing vector type equals considering names and metadata

2019-08-18 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6288: - Summary: [Java] Implement TypeEqualsVisitor comparing vector type equals considering names and metadata Key: ARROW-6288 URL: https://issues.apache.org/jira/browse/ARROW-6288

[jira] [Created] (ARROW-6265) [Java] Avro adapter implement Array/Map/Fixed type

2019-08-15 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6265: - Summary: [Java] Avro adapter implement Array/Map/Fixed type Key: ARROW-6265 URL: https://issues.apache.org/jira/browse/ARROW-6265 Project: Apache Arrow Issue Type: Sub

[jira] [Created] (ARROW-6250) [Java] Implement ApproxEqualsVisitor comparing approx for floating point

2019-08-15 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6250: - Summary: [Java] Implement ApproxEqualsVisitor comparing approx for floating point Key: ARROW-6250 URL: https://issues.apache.org/jira/browse/ARROW-6250 Project: Apache Arrow

[jira] [Created] (ARROW-6249) [Java] Remove useless class ByteArrayWrapper

2019-08-15 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6249: - Summary: [Java] Remove useless class ByteArrayWrapper Key: ARROW-6249 URL: https://issues.apache.org/jira/browse/ARROW-6249 Project: Apache Arrow Issue Type: Improvement

Re: [DISCUSS][JAVA] Make FixedSizedListVector inherit from ListVector

2019-08-14 Thread Ji Liu
setBuffer, I am fine with the approach you suggested, BaseRepeatedVector/ListVector and FixedSizeListVector both inherit from new interface(BaseListVector) they can be used interchangeably in dictionary encoding.Let see if others have different opinions. Thank

Re: [ANNOUNCE] New Arrow PMC member: Sebastien Binet

2019-08-14 Thread Ji Liu
Congrats Sebastian! -- From:Micah Kornfield Send Time:2019年8月15日(星期四) 10:46 To:dev@arrow.apache.org Subject:Re: [ANNOUNCE] New Arrow PMC member: Sebastien Binet Congrats. Well deserved. On Wednesday, August 14, 2019, paddy horan

Re: [Java] CI builds failing on master

2019-08-14 Thread Ji Liu
Hi, Wes, as described in JIRA, this was introduced by our recent two patches, I have just submitted a PR[1] to fix this. Thanks for tracking this issue. Thanks, Ji Liu [1] https://github.com/apache/arrow/pull/5090 -- From:Wes

[jira] [Created] (ARROW-6234) [Java] ListVector hashCode() is not correct

2019-08-14 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6234: - Summary: [Java] ListVector hashCode() is not correct Key: ARROW-6234 URL: https://issues.apache.org/jira/browse/ARROW-6234 Project: Apache Arrow Issue Type: Bug

[jira] [Created] (ARROW-6218) [Java] Add UINT type test in integration to avoid potential overflow

2019-08-12 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6218: - Summary: [Java] Add UINT type test in integration to avoid potential overflow Key: ARROW-6218 URL: https://issues.apache.org/jira/browse/ARROW-6218 Project: Apache Arrow

Re: [DISCUSS][JAVA] Make FixedSizedListVector inherit from ListVector

2019-08-11 Thread Ji Liu
Thanks Jacques, to avoid complex call paths for getObject, should keep getObject for both classes. I'll also checked for other methods. Thanks, Ji Liu -- From:Jacques Nadeau Send Time:2019年8月11日(星期日) 21:43 To:dev ; J

[jira] [Created] (ARROW-6200) [Java] Method getBufferSizeFor in BaseRepeatedValueVector/ListVector not correct

2019-08-11 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6200: - Summary: [Java] Method getBufferSizeFor in BaseRepeatedValueVector/ListVector not correct Key: ARROW-6200 URL: https://issues.apache.org/jira/browse/ARROW-6200 Project: Apache

[jira] [Created] (ARROW-6199) [Java] Avro adapter avoid potential resource leak.

2019-08-11 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6199: - Summary: [Java] Avro adapter avoid potential resource leak. Key: ARROW-6199 URL: https://issues.apache.org/jira/browse/ARROW-6199 Project: Apache Arrow Issue Type: Bug

Re: [DISCUSS][JAVA] Make FixedSizedListVector inherit from ListVector

2019-08-10 Thread Ji Liu
do you think? Thanks, Ji Liu -- From:Micah Kornfield Send Time:2019年8月11日(星期日) 08:47 To:dev ; Ji Liu Subject:Re: [DISCUSS][JAVA] Make FixedSizedListVector inherit from ListVector Hi Ji Liu, I think have a common interface/base

Re: [Java] Arrow PR queue build up?

2019-08-10 Thread Ji Liu
Micah > > > > On Thursday, August 8, 2019, Wes McKinney wrote: > > > >> hi folks, > >> > >> Liya Fan and Ji Liu have about 24 open Java PRs between them if I > >> counted right -- it seems like the project is having a hard time > &g

[DISCUSS][JAVA] Make FixedSizedListVector inherit from ListVector

2019-08-10 Thread Ji Liu
, should we make former extends the latter to reduce the plenty duplicated logic in these two and writer/reader classes? Thanks, Ji Liu [1] https://issues.apache.org/jira/browse/ARROW-1175[2]https://github.com/apache/arrow/pull/4972

[jira] [Created] (ARROW-6194) [Java] Make DictionaryEncoder non-static making it easy to extend and reuse

2019-08-10 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6194: - Summary: [Java] Make DictionaryEncoder non-static making it easy to extend and reuse Key: ARROW-6194 URL: https://issues.apache.org/jira/browse/ARROW-6194 Project: Apache Arrow

Re: [ANNOUNCE] New Arrow PMC member: Micah Kornfield

2019-08-09 Thread Ji Liu
Congrats Micah! Well deserved. Thanks, Ji Liu -- From:paddy horan Send Time:2019年8月9日(星期五) 23:31 To:dev@arrow.apache.org Subject:Re: [ANNOUNCE] New Arrow PMC member: Micah Kornfield Congrats Micah! Get Outlook for iOS<ht

[jira] [Created] (ARROW-6175) [Java] Fix MapVector#getMinorType and extend AbstractContainerVector addOrGet complex vector API

2019-08-08 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6175: - Summary: [Java] Fix MapVector#getMinorType and extend AbstractContainerVector addOrGet complex vector API Key: ARROW-6175 URL: https://issues.apache.org/jira/browse/ARROW-6175

[jira] [Created] (ARROW-6160) [Java] AbstractStructVector#getPrimitiveVectors fails to work with complex child vectors

2019-08-07 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6160: - Summary: [Java] AbstractStructVector#getPrimitiveVectors fails to work with complex child vectors Key: ARROW-6160 URL: https://issues.apache.org/jira/browse/ARROW-6160 Project

[jira] [Created] (ARROW-6145) [Java] UnionVector created by MinorType#getNewVector could not keep field type info properly

2019-08-06 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6145: - Summary: [Java] UnionVector created by MinorType#getNewVector could not keep field type info properly Key: ARROW-6145 URL: https://issues.apache.org/jira/browse/ARROW-6145 Project

[jira] [Created] (ARROW-6118) [Java] Replace google Preconditions with Arrow Preconditions

2019-08-02 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6118: - Summary: [Java] Replace google Preconditions with Arrow Preconditions Key: ARROW-6118 URL: https://issues.apache.org/jira/browse/ARROW-6118 Project: Apache Arrow Issue

[jira] [Created] (ARROW-6097) [Java] Avro adapter implement unions type

2019-08-01 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6097: - Summary: [Java] Avro adapter implement unions type Key: ARROW-6097 URL: https://issues.apache.org/jira/browse/ARROW-6097 Project: Apache Arrow Issue Type: Sub-task

[jira] [Created] (ARROW-6083) [Java] Refactor Jdbc adapter consume logic

2019-07-31 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6083: - Summary: [Java] Refactor Jdbc adapter consume logic Key: ARROW-6083 URL: https://issues.apache.org/jira/browse/ARROW-6083 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-6079) [Java] Implement/test UnionFixedSizeListWriter for FixedSizeListVector

2019-07-31 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6079: - Summary: [Java] Implement/test UnionFixedSizeListWriter for FixedSizeListVector Key: ARROW-6079 URL: https://issues.apache.org/jira/browse/ARROW-6079 Project: Apache Arrow

[jira] [Created] (ARROW-6078) [Java] Implement dictionary-encoded subfields for List type

2019-07-30 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6078: - Summary: [Java] Implement dictionary-encoded subfields for List type Key: ARROW-6078 URL: https://issues.apache.org/jira/browse/ARROW-6078 Project: Apache Arrow Issue

[jira] [Created] (ARROW-6035) [Java] Avro adapter support convert nullable value

2019-07-25 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6035: - Summary: [Java] Avro adapter support convert nullable value Key: ARROW-6035 URL: https://issues.apache.org/jira/browse/ARROW-6035 Project: Apache Arrow Issue Type: Sub

[jira] [Created] (ARROW-6022) [Java] Support equals API in ValueVector to compare two vectors equal

2019-07-24 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6022: - Summary: [Java] Support equals API in ValueVector to compare two vectors equal Key: ARROW-6022 URL: https://issues.apache.org/jira/browse/ARROW-6022 Project: Apache Arrow

[jira] [Created] (ARROW-6020) [Java] Refactor ByteFunctionHelper#hash with new added ArrowBufHasher

2019-07-23 Thread Ji Liu (JIRA)
Ji Liu created ARROW-6020: - Summary: [Java] Refactor ByteFunctionHelper#hash with new added ArrowBufHasher Key: ARROW-6020 URL: https://issues.apache.org/jira/browse/ARROW-6020 Project: Apache Arrow

  1   2   >