Re: Support for numpy matrix

2019-03-30 Thread Wes McKinney
hi Mitar, Let's discuss further on JIRA? It's best to keep all the information about the issue in one place. Thanks On Sat, Mar 30, 2019 at 7:42 PM Mitar wrote: > > Hi! > > I added: > > serialization_context.register_type( > np.matrix, 'np.matrix', > custom_serializer=_serialize_numpy_a

Re: Support for numpy matrix

2019-03-30 Thread Mitar
Hi! I added: serialization_context.register_type( np.matrix, 'np.matrix', custom_serializer=_serialize_numpy_array_list, custom_deserializer=_deserialize_numpy_array_list) But it did not help. Probably also because np.matrix is a subclas of np.ndarray anyway. So no change here. An i

Re: Support for numpy matrix

2019-03-30 Thread Philipp Moritz
Hey Mitar, It might be as simple as adding a handler here: https://github.com/apache/arrow/blob/master/python/pyarrow/serialization.py#L300 Do you want to try that? -- Philipp. On Sat, Mar 30, 2019 at 3:22 PM Mitar wrote: > Hi! > > I do not know where to start looking into this? Not sure if I

Re: Support for numpy matrix

2019-03-30 Thread Wes McKinney
Let's discuss on the JIRA issue. AFAIK we have no unit tests relating to numpy.matrix so most likely this was an accidental "feature". If there is no other volunteer to help on the issue it's hard to predict when it will get done. numpy.matrix isn't used by very many people On Sat, Mar 30, 2019 at

Re: Support for numpy matrix

2019-03-30 Thread Mitar
Hi! I do not know where to start looking into this? Not sure if I have enough knowledge about arrow to be able to make a PR. Miar On Sat, Mar 30, 2019 at 3:17 PM Wes McKinney wrote: > > hi Mitar, > > I see you reported the issue on October 2 and no one has volunteered > to fix it yet. Are you

Re: Support for numpy matrix

2019-03-30 Thread Wes McKinney
hi Mitar, I see you reported the issue on October 2 and no one has volunteered to fix it yet. Are you up to submit a PR? Thanks Wes On Sat, Mar 30, 2019 at 5:14 PM Mitar wrote: > > Hi! > > It seems numpy's matrix is not supported in recent versions of pyarrow: > > https://issues.apache.org/jira

Support for numpy matrix

2019-03-30 Thread Mitar
Hi! It seems numpy's matrix is not supported in recent versions of pyarrow: https://issues.apache.org/jira/browse/ARROW-3399 Any ideas why this would be happening? Mitar -- http://mitar.tnode.com/ https://twitter.com/mitar_m

Re: [VOTE] Release Apache Arrow 0.13.0 - RC4

2019-03-30 Thread Wes McKinney
I just reopened https://issues.apache.org/jira/browse/ARROW-4301 There is a problem with updating the Gandiva Java POM file that also affected 0.12.0. I do not think it is a release blocker On Fri, Mar 29, 2019 at 11:42 AM Wes McKinney wrote: > > There need to be a net of 3 binding +1 votes. Rel

Re: FPGA support for Apache Arrow

2019-03-30 Thread Wes McKinney
Thank you, this is helpful. On the page alignment issue, is it enough to ensure that each buffer (according to the IPC protocol) has a size with a certain multiple? This might be something we can make configurable on the IPC write path on the C++ side. There is nothing fundamentally illegal in the

Re: FPGA support for Apache Arrow

2019-03-30 Thread Dimitris Lekkas
Hi, Basically, fletcher has emphasized on reading an Arrow schema and generating a hardware interface along with a template for the functional part of the accelerator. That way the FPGAs are aware for arrow-backed data. On the other hand, we do not work on enabling FPGAs to interpret arrow format

Re: [Discuss] Benchmarking infrastructure

2019-03-30 Thread Antoine Pitrou
Le 29/03/2019 à 16:06, Wes McKinney a écrit : > >> * How to make it available to all developers? Do we want to integrate into >> CI or not? > > I'd like to eventually have a bot that we can ask to run a benchmark > comparison versus master. Reporting on all PRs automatically might be > quite a