RE: [C++][Go] CGO For Dataset API Integration

2021-08-23 Thread Matthew Topol
om: Antoine Pitrou > Sent: Monday, August 23, 2021 2:00 PM > To: dev@arrow.apache.org > Subject: Re: [C++][Go] CGO For Dataset API Integration > > > Le 23/08/2021 à 19:53, Matthew Topol a écrit : >> The only thing I don't like it being a private module in the Go >> impl

Re: [C++][Go] CGO For Dataset API Integration

2021-08-23 Thread Antoine Pitrou
ile (the top of the module) it knows that it only needs the go/arrow directory tree for the module and as such it doesn't clone the entire git repo. -Original Message- From: Antoine Pitrou Sent: Monday, August 23, 2021 2:00 PM To: dev@arrow.apache.org Subject: Re: [C++][Go]

RE: [C++][Go] CGO For Dataset API Integration

2021-08-23 Thread Matthew Topol
h to the go.mod file (the top of the module) it knows that it only needs the go/arrow directory tree for the module and as such it doesn't clone the entire git repo. -Original Message- From: Antoine Pitrou Sent: Monday, August 23, 2021 2:00 PM To: dev@arrow.apache.org Subject: R

Re: [C++][Go] CGO For Dataset API Integration

2021-08-23 Thread Antoine Pitrou
Le 23/08/2021 à 19:53, Matthew Topol a écrit : The only thing I don't like it being a private module in the Go implementation is distribution. For native go code, consumers can just perform `go get` and have it work. But for this interface, it would require both consumers of the module and a

RE: [C++][Go] CGO For Dataset API Integration

2021-08-23 Thread Matthew Topol
1:24 PM To: dev@arrow.apache.org Subject: Re: [C++][Go] CGO For Dataset API Integration Le 23/08/2021 à 19:16, Matthew Topol a écrit : > Unfortunately, Go currently can only integrate with C++ libraries through a C > interface. There does exist SWIG which is a generator for creating i

Re: [C++][Go] CGO For Dataset API Integration

2021-08-23 Thread Antoine Pitrou
Le 23/08/2021 à 19:16, Matthew Topol a écrit : Unfortunately, Go currently can only integrate with C++ libraries through a C interface. There does exist SWIG which is a generator for creating interface code between Go and C++, but ultimately it's just automating the creation of a C interface

RE: [C++][Go] CGO For Dataset API Integration

2021-08-23 Thread Matthew Topol
ementation, only for using the Dataset API stuff (and potentially the compute library). --Matt -Original Message- From: Antoine Pitrou Sent: Monday, August 23, 2021 12:56 PM To: dev@arrow.apache.org Subject: Re: [C++][Go] CGO For Dataset API Integration Le 23/08/2021 à 18:22, Matt

Re: [C++][Go] CGO For Dataset API Integration

2021-08-23 Thread Antoine Pitrou
Le 23/08/2021 à 18:22, Matthew Topol a écrit : That's a fair point, and part of the work I've done so far is a local Go implementation of at least consuming the C data interface. It will also eventually involve creating the necessary implementation to produce the C-Data interface too. But sp

RE: [C++][Go] CGO For Dataset API Integration

2021-08-23 Thread Matthew Topol
e same, aside from performance). --Matt -Original Message- From: Antoine Pitrou Sent: Monday, August 23, 2021 12:00 PM To: dev@arrow.apache.org Subject: Re: [C++][Go] CGO For Dataset API Integration Hi Matt, As the name suggests, the C data interface is not a *programming* interface. I

Re: [C++][Go] CGO For Dataset API Integration

2021-08-23 Thread Antoine Pitrou
Hi Matt, As the name suggests, the C data interface is not a *programming* interface. It is a data sharing convention which relies on the existence of dedicated endpoints to produce or consume the C data structures. For example in Arrow C++, there is this set of APIs: https://arrow.apache

[C++][Go] CGO For Dataset API Integration

2021-08-23 Thread Matthew Topol
Hey All, So I've been working on a use case where I needed to be able to use the Dataset API from Golang and instead of trying to port all of it to Golang (which would require porting the Compute side too) I decided to create a proof of concept using CGO to just call into the existing C++ code