Also, I'd like to clarify: - [David]: is it a good idea to use cgo for Go-Python interop? - [Michael]: no. better with pipe or RPC
I'm wrong about this. A conversation after the meeting clarified a misunderstanding I had about Go->Python calls specifically. Both Go->Python and Python->Go with cgo may very well be preferable to pipe/RPC in many circumstances. :) On Thursday, November 23, 2023 at 3:40:35 AM UTC-5 Sebastien Binet wrote: > On Thu Nov 23, 2023 at 01:24 CET, Eli Bendersky wrote: > > On Wed, Nov 22, 2023 at 11:31 AM Sebastien Binet > > <sebasti...@cern.ch> > > wrote: > > > > > Hi there, > > > > > > In this week "compiler minutes" [1], one can read: > > > > > > """ > > > - Go on future platforms (RAM efficiency. NUMA?) > > > - (maybe) Go-Python interop for AI-powered applications > > > - [David]: is it a good idea to use cgo for Go-Python interop? > > > - [Michael]: no. better with pipe or RPC > > > """ > > > > > > Would it be possible to have a bit more informations ? > > > What kind of interop is it ? Exchanging binary data ? On disk ? > > > Establishing a protobuf-based-like standard ? > > > Something else ? > > > > > > > All of it, maybe :-) > > We're just exploring the issue, throwing ideas around. There are many > > potential options, each with its own tradeoffs in terms of performance > > vs. effort. > > depending on the timescale, one could also imagine having a Go-based > python VM (e.g. github.com/go-python/gpython) that can run a limited set > of python modules (even the C-based ones, like pypy did at some point ?). > alternatively, "just" rely on pickle-based, Apache Arrow-based or numpy > array-based exchanged data. > > for Arrow and numpy, there are already packages that do offer a fair > amount of interop: > > - https://github.com/apache/arrow/tree/main/go > - https://github.com/sbinet/npyio/ (shameless plug.) > > (we could imagine also adding some buffer protocol implementation à la > PEP-3118) > > for pickle, gpython has support up to protocol=3, and gopickle seems to > have support for up to 5: > - https://github.com/nlpodyssey/gopickle > (adding support for array.array to gopickle was relatively straightforward) > > -s > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/cf461932-52bb-4ac1-9690-053bec7e432an%40googlegroups.com.
