Hi Jan, I'm going to go out on a limb here and suggest looking at using Go's `embed` feature?
https://blog.jetbrains.com/go/2021/06/09/how-to-use-go-embed-in-go-1-16/ I have not tried it with C libraries so there may be numerous reasons why it may not work for your needs. For example, I do not know what is required to *"install"* the libraries so that might be your sticking point that embed would not address. But if building on the user's machine is the real problem and you can distribute pre-build binaries then this might work for you. Basically Go reads a `//go:embed` comment, converts your files into Go source code, compiles that code, and then provides a package that allows you to write those files to disk from an your Go app before you need to load the libraries. Maybe this will work for you? If yes, would love to hear back how it worked out. -Mike On Monday, October 16, 2023 at 8:40:54 AM UTC-4 Bruno Albuquerque wrote: > I guess you switched things here. Shared libraries (.so) need to be > available at engine. Static libraries (.a) are bagged into the binary. > > -Bruno > > > On Sun, Oct 15, 2023, 3:55 PM Jan <pfe...@gmail.com> wrote: > >> Thanks Tamas, this is useful information. >> >> One of my libraries is using a `.a` library -- as opposed to `.so`, which >> is another level of complexity, since the library has to be available in >> runtime, not only in compile time -- and I'm going to follow your >> "incantation" suggestion. >> >> >> On Sunday, October 15, 2023 at 7:55:55 AM UTC+2 Tamás Gulácsi wrote: >> >>> Neither I see a convenient way. >>> BUT if you add a .go file into the directories where your precompiled >>> libraries live, >>> then "go get" will download them too (and only those dirs that have .go >>> files in it). >>> >>> So your next mission is to prepare the right #cgo CFLAGS LDFLAGS >>> incantations to use those libraries. >>> >>> Jan a következőt írta (2023. október 14., szombat, 8:37:48 UTC+2): >>> >>>> Thanks Tamás, I may not be understanding correctly, but after taking a >>>> look at github.com/godror/godror, and the odpi subdirectory, >>>> I see it is including all the `.c` files on the fly >>>> <https://github.com/godror/godror/blob/main/odpi/embed/dpi.c>. >>>> >>>> A couple of reasons immediately come to mind, that make this not a >>>> generically valid option: >>>> >>>> * ODPI library is all C code (see src >>>> <https://github.com/godror/godror/tree/main/odpi/src>) so it works >>>> including in Go: my dependencies are C++/Rust code, for which I write a >>>> small C wrapper (or for Rust just `extern "C"`). Also architecture >>>> dependent compilation is different in C++/C/Rust code ... >>>> * The C++ libraries I'm including have sub-dependencies of themselves >>>> (one of which is llvm). It uses Bazel to organize it, and to manually move >>>> all the required C++ files to a directory would be years of work :) Plus >>>> would require me to slave to maintain things in sync. >>>> * The C++ libraries take hours to compile ... I don't want to impose >>>> this to users of my libraries. >>>> >>>> I think the only way to work this out is distributing the pre-compiled >>>> C++/Rust libraries, so the Go simply refer to them (and we get the fast >>>> compilation times from Go). But then, how to best distribute them in an >>>> automatic fashion, so that users don't need to one by one figure out how >>>> to >>>> install them (and clean up them later if they are no longer using...) ? >>>> >>>> Or maybe there is another convenient way I'm not seeing ? >>>> >>>> >>>> On Thursday, October 12, 2023 at 6:39:34 PM UTC+2 Tamás Gulácsi wrote: >>>> >>>>> Can't you build (make go build for you) those libraries? >>>>> For example, see github.com/godror/godror just includes the sources >>>>> of that third party library in an "odpi" subdir, and with >>>>> ``` >>>>> /* >>>>> #cgo CFLAGS: -I./odpi/include -I./odpi/src -I./odpi/embed >>>>> >>>>> #include "dpi.c" >>>>> >>>>> */ >>>>> import "C" >>>>> ``` >>>>> it is compiled automatically. >>>>> >>>>> >>>>> Caveat: for "go get" to download a directory, it must include a sth.go >>>>> file ("require.go" in the odpi/* subdirs). >>>>> But it may work that your precompiled libs in a subdir, with a mock >>>>> .go file gets downloaded. >>>>> But how will it found by your lib/app ? >>>>> >>>>> Tamás >>>>> >>>>> >>>>> Jan a következőt írta (2023. október 12., csütörtök, 8:14:41 UTC+2): >>>>> >>>>>> Thanks Richard. Indeed, as you pointed out the downside is the >>>>>> bloating of the git repo, but it makes sense. >>>>>> >>>>>> But does the user have to manually clone the repository and move the >>>>>> `.a` file to, let's say, /usr/local/lib, or does a simple `go get` >>>>>> magically does everything ? >>>>>> >>>>>> >>>>>> On Thursday, October 12, 2023 at 2:29:21 AM UTC+2 Richard Wilkes >>>>>> wrote: >>>>>> >>>>>>> It isn't a great solution, but I currently include the built library >>>>>>> files and necessary headers in the git repo alongside the Go code. You >>>>>>> can >>>>>>> see an example here >>>>>>> https://github.com/richardwilkes/unison/tree/main/internal/skia >>>>>>> where I include the skia library I built for use in my UI framework, >>>>>>> unison. >>>>>>> >>>>>>> The main downside of this is bloating the git repo with the binary >>>>>>> .a and .dll files... but I've not found a better way to handle it. >>>>>>> glfw, >>>>>>> which unison also depends upon, does something very similar. >>>>>>> >>>>>>> - Rich >>>>>>> >>>>>>> On Wednesday, October 11, 2023 at 2:58:23 AM UTC-7 Jan wrote: >>>>>>> >>>>>>>> hi, >>>>>>>> >>>>>>>> I'm developing a couple of ML framework/libraries for Go that >>>>>>>> depend on C/C++ code. Once C-libraries dependencies are installed, the >>>>>>>> CGO >>>>>>>> integration work great. >>>>>>>> >>>>>>>> Now, for end-users that just want to use these Go libraries, having >>>>>>>> to figure out how to manually build and install those C/C++/Rust >>>>>>>> dependencies is a hassle -- sadly each one with a different type of >>>>>>>> build >>>>>>>> system. >>>>>>>> >>>>>>>> I offer pre-built `.tgz` files (for a limited set of architectures) >>>>>>>> with the required `.h` and `.a/.so` files along the releases, which >>>>>>>> facilitates. But it's still a hassle to install -- and no >>>>>>>> auto-uninstall if >>>>>>>> someone is no longer using the Go module. >>>>>>>> >>>>>>>> I was wondering if others have figured out how to handle this in a >>>>>>>> nicer way. Is there a recommended way to distribute prebuilt CGO >>>>>>>> dependencies ? >>>>>>>> >>>>>>>> I like how Python wheels (`.whl` files) solve the issue, by >>>>>>>> including the pre-built libraries in a sub-directory of the python >>>>>>>> library >>>>>>>> installation. I was hoping there would be a similar way (maybe with a >>>>>>>> separate tool) to integrate with `go.mod`. >>>>>>>> >>>>>>>> Any pointers, thoughts or suggestions are very appreciated. >>>>>>>> >>>>>>>> Thanks! >>>>>>>> Jan >>>>>>>> >>>>>>>> ps: While searching I saw similar questions, but none that exactly >>>>>>>> answered this aspect of distribution. Just in case, apologies if it's >>>>>>>> a >>>>>>>> duplicate question. >>>>>>>> >>>>>>> -- >> > You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts...@googlegroups.com. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/a00b3a7b-200d-44ce-8060-913d9ac3c6cen%40googlegroups.com >> >> <https://groups.google.com/d/msgid/golang-nuts/a00b3a7b-200d-44ce-8060-913d9ac3c6cen%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/ca2d8979-f741-4b81-9099-8672bc5e2312n%40googlegroups.com.
