Re: Meetup in SF, Additional Speakers?

2018-03-27 Thread Jacques Nadeau
Thanks for enthusiasm (both on list and off list). I think we're going to try to do a second meetup later in the summer in the South Bay as well. Will get back to everyone shortly. Great to see all the interest. thanks, Jacques On Tue, Mar 27, 2018 at 1:00 PM, Sourav Mazumder < sourav.mazumde...@

Re: Using Plasma with xnd

2018-03-27 Thread Wes McKinney
> But, how do I access either the protected mutable_data_ or the inline > mutable_data from my this wrapper? I may be missing something, but you need to use Buffer::mutable_data() somewhere in the wrapper. So either provide a function in the wrapper that calls it, or invoke it during some init s

Re: Using Plasma with xnd

2018-03-27 Thread Philipp Moritz
Also, if you provide me with full instructions on how to reproduce the linking error, I'm happy to look if I can figure out what is going on. On Tue, Mar 27, 2018 at 1:25 PM, Philipp Moritz wrote: > Hey Saul, > > The Buffer returned by Plasma's Create is not owning the memory, the > memory manag

Re: Using Plasma with xnd

2018-03-27 Thread Philipp Moritz
Hey Saul, The Buffer returned by Plasma's Create is not owning the memory, the memory management happens via reference counting through the explicit PlasmaClient::Release call. The reason why we now return a buffer instead of a plain pointer and length is so we can support both CPU (MutableBuffer)

Re: Using Plasma with xnd

2018-03-27 Thread Saul Shanabrook
Yeah, I am writing C++ code that provides a C wrapper around this. But, how do I access either the protected mutable_data_ or the inline mutable_data from my this wrapper? On Tue, Mar 27, 2018 at 10:52 AM Wes McKinney wrote: > @Kou, we removed that bit, so Buffer::mutable_data is always inline

Re: Meetup in SF, Additional Speakers?

2018-03-27 Thread Sourav Mazumder
Hi Wes, Here goes some inputs on these 2 usecases. 1. The first use case is around calling Watson Services in a scalable manner using a custom Rest Data Source https://developer.ibm.com/dwblog/2018/distributed-rest-calls-to-watson-services-using-rest-data-source-on-apache-spark/. The idea is to m

Re: Meetup in SF, Additional Speakers?

2018-03-27 Thread Wes McKinney
hi Sourav, Do you have prior references for either of these topics / use cases? I had not heard about them before. Thanks, Wes On Tue, Mar 27, 2018 at 2:12 PM, Sourav Mazumder wrote: > Hi Jacques, > > I can talk about on either of these 2 topics - > > 1. Using Arrow with IBM Watson Studio for v

Re: Meetup in SF, Additional Speakers?

2018-03-27 Thread Sourav Mazumder
Hi Jacques, I can talk about on either of these 2 topics - 1. Using Arrow with IBM Watson Studio for vectorized query processing on large volume of data 2. Using Arrow for NetCDF data format for supporting scientific data processing Regards, Sourav Mazumder Data Science Center of Competency IBM

Re: Meetup in SF, Additional Speakers?

2018-03-27 Thread Bryan Cutler
Hi Jacques, I could talk about some of the integration with Spark, if you have room for another. Thanks, Bryan On Tue, Mar 27, 2018 at 9:51 AM, Atul Dambalkar wrote: > Hi Jacques, > > If it makes sense, I can certainly talk about the JDBC Adapter work I have > been doing and also some of the f

RE: Meetup in SF, Additional Speakers?

2018-03-27 Thread Atul Dambalkar
Hi Jacques, If it makes sense, I can certainly talk about the JDBC Adapter work I have been doing and also some of the future enhancements/related bits of work that could happen surrounding that. Please let me know if this would be useful for the audience. Regards, -Atul -Original Messag

Meetup in SF, Additional Speakers?

2018-03-27 Thread Jacques Nadeau
Hey All, It looks like Thumbtack crew offered to host an Arrow meetup for the Arrow community in San Francisco and one of my colleagues set it up on Meetup. Sidd and I have been volunteered to do talks but I would really love to have one or two other speakers as well. Anyone going to be in San Fra

[jira] [Created] (ARROW-2359) Type objects produced by DataType factory are not thread safe

2018-03-27 Thread Anton Shmigirilov (JIRA)
Anton Shmigirilov created ARROW-2359: Summary: Type objects produced by DataType factory are not thread safe Key: ARROW-2359 URL: https://issues.apache.org/jira/browse/ARROW-2359 Project: Apache A

Re: Using Plasma with xnd

2018-03-27 Thread Wes McKinney
@Kou, we removed that bit, so Buffer::mutable_data is always inline now. @Saul, you will want to create a C wrapper for arrow::Buffer so you can manage memory lifetime from C. Depending on the kind of Buffer you have, it may not be as simple as calling free(...) on its internal data. - Wes On Tu

Re: Using Plasma with xnd

2018-03-27 Thread Kouhei Sutou
Hi, It may be wrong but you may need to add -DNDEBUG flag to C compiler. Thanks, -- kou In "Re: Using Plasma with xnd" on Tue, 27 Mar 2018 13:41:49 +, Saul Shanabrook wrote: > Ah, I see I can use garrow_buffer_get_mutable_data from c_glib. > > On Tue, Mar 27, 2018 at 9:29 AM Saul Sh

Re: Using Plasma with xnd

2018-03-27 Thread Saul Shanabrook
Ah, I see I can use garrow_buffer_get_mutable_data from c_glib. On Tue, Mar 27, 2018 at 9:29 AM Saul Shanabrook wrote: > I am in the process of changing over my C wrapper to work with Arrow > 0.9.0. One change is that the plasma APIs now use arrow::Buffer instead of > of raw pointers, in Create

Re: Using Plasma with xnd

2018-03-27 Thread Saul Shanabrook
I am in the process of changing over my C wrapper to work with Arrow 0.9.0. One change is that the plasma APIs now use arrow::Buffer instead of of raw pointers, in Create and Get. In my C wrapper for Create, I need to get the mutable data pointer from the arrow::Buffer, after it is created. Here i