Re: BigQuery Storage API now supports Arow

2019-07-27 Thread Jonathan Chiang
Hi Micah, That’s awesome!! It’s pretty surreal to request a feature from google and have it built out. Thanks, Jonathan > On Jul 26, 2019, at 8:43 PM, Micah Kornfield wrote: > > Hi Arrow Dev, > As a follow-up to an old thread [1] on working with BigQuery and Arrow. I > just wanted to share

Any formal plans for mxnet/xgboost and arrow collaboration?

2019-03-19 Thread Jonathan Chiang
Hi, I was curious if there was any concrete roadmap to build compatibility between these Apache frameworks. Would love to do data processing in python/R in arrow for deep learning or tabular data model building. My use case is model building and interoperability between R and Python users at

Re: Arrow and R benchmark

2019-02-28 Thread Jonathan Chiang
I'm optimistic that BQ will > offer native Arrow export at some point in the future, perhaps after > we reach a "1.0.0" release > > - Wes > > > On Sat, Feb 23, 2019 at 12:17 PM Jonathan Chiang > wrote: > > > > Hi Micah, > > > > Yes

Re: Arrow and R benchmark

2019-02-23 Thread Jonathan Chiang
; be very interested. >> > > >> > > One hurdle for BigQuery is that my understanding is that Google has >> > > policies in place that make it more difficult to take on external >> > > library dependencies in a sensitive system like Dremel / BigQuery. So &g

Re: Arrow Flight protocol/API questions

2019-02-12 Thread Jonathan Chiang
Would zeromq make more sense than gRPC? Thanks, Jonathan > On Feb 12, 2019, at 12:48 PM, Antoine Pitrou wrote: > > > Hi David, > > I think allowing to send application-specific ancillary data in addition > to Arrow data makes sense. > > (I'm also wondering whether the choice of gRPC is app

Re: Arrow and R benchmark

2019-02-04 Thread Jonathan Chiang
PM Wes McKinney wrote: > hi Jonathan, > On Sat, Nov 24, 2018 at 6:19 PM Jonathan Chiang > wrote: > > > > Hi Wes and Romain, > > > > I wrote a preliminary benchmark for reading and writing different file > types from R into arrow, borrowed some code from Hadley.

Re: Enabling an installation path for Arrow R users

2019-01-03 Thread Jonathan Chiang
Hi, Long term, I believe getting an arrow package onto cran would be most useful for R users. Building arrow in R on Mac was easier than Linux for me. I was still having trouble installing after spending a couple hours or so. Typically if you can install.package from cran is most convenient

Re: Arrow and R benchmark

2018-11-24 Thread Jonathan Chiang
. Was unable to build on linux. On Thu, Nov 15, 2018 at 7:50 PM Jonathan Chiang wrote: > I'll go through that python repo and see what I can do. > > Thanks, > Jonathan > > On Thu, Nov 15, 2018 at 1:55 PM Wes McKinney wrote: > >> I would suggest starting an r/b

Re: Arrow and R benchmark

2018-11-15 Thread Jonathan Chiang
> > > Romain > > > > > Le 15 nov. 2018 à 22:16, Wes McKinney a écrit : > > > > > > Adding dev@arrow.apache.org > > >> On Thu, Nov 15, 2018 at 4:13 PM Jonathan Chiang > wrote: > > >> > > >> Hi, >