Hello Ian, Yes, I think it is time I explain the 'why' of my inquiries into this. :-)
My use case is this: Go, with its fast startup, pretty fast execution and pretty small memory footprint, as well as it's ability to deploy as a single binary, makes it a great language for cloud function-as-a-service (FAAS). My specific FAAS use case is for static, read-only databases: I am looking at embedding modest sized (few GB to 10s of GB) read-only databases in the Go binary. This makes it possible to avoid the cost/complication of either using a cloud db, or storing the db in (in the case of AWS) on a file system like EFS (NFS), which the lambda has to mount. The databases are only updated every couple of months / yearly. This is perhaps rather an obscure use case, but one that I think a number of people would want to take advantage of. >Note that such files are going to take a long time to create and will be unwieldy to use. Agreed. On my older desktop (Dell 7010 Kubuntu 20.10 4core i5 16GM) it takes 45s to compile a trivial Go program that embeds three 500GB files. Actually, this is no different from those who want the simplicity and cost savings of serving an entire static web site from embedded files. The primary difference is the scale. Thanks, Glen On Tuesday, September 21, 2021 at 1:28:12 PM UTC-4 Ian Lance Taylor wrote: > On Tue, Sep 21, 2021 at 6:23 AM Glen Newton <glen....@gmail.com> wrote: > > > > Looking at https://groups.google.com/g/golang-codereviews/c/xkHLQHidF5s > and https://github.com/golang/go/issues/9862 and the code to which they > refer, it seems to me that the limit is 2GB independent of platform (is > this true?), as the linker is limited to 2e9. > > Again, should have done more of my homework! :-) > > > > > https://github.com/golang/go/blob/master/src/cmd/internal/obj/objfile.go#L305 > > > > // cutoff is the maximum data section size permitted by the linker > > // (see issue #9862). const cutoff = 2e9 // 2 GB (or so; looks better in > errors than 2^31) > > const cutoff = 2e9 // 2 GB (or so; looks better in errors than 2^31) > > > > The comment indicates that this is a limit in the linker; is this a > limit in the elf format? If No, could the linker et al be modified to > accept larger static data? > > You're right: it does look like cmd/link imposes a 2G total limit. > This is not a limitation of the 64-bit ELF format. It should be > possible to make it larger. > > Note that such files are going to take a long time to create and will > be unwieldy to use. While it should be possible to increase the size, > I would not be surprised if you hit other limits fairly quickly. > > Ian > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/9aee61f1-be26-45a8-96f6-b06ff65983ban%40googlegroups.com.
