Hello, Looking at https://groups.google.com/g/golang-codereviews/c/xkHLQHidF5s and https://github.com/golang/go/issues/9862 and the code to which they refer, it seems to me that the limit is 2GB independent of platform (is this true?), as the linker is limited to 2e9. Again, should have done more of my homework! :-)
https://github.com/golang/go/blob/master/src/cmd/internal/obj/objfile.go#L305 // cutoff is the maximum data section size permitted by the linker // (see issue #9862). const cutoff = 2e9 // 2 GB (or so; looks better in errors than 2^31) const cutoff = 2e9 // 2 GB (or so; looks better in errors than 2^31) The comment indicates that this is a limit in the linker; is this a limit in the elf format? If No, could the linker et al be modified to accept larger static data? Thanks, Glen On Monday, September 20, 2021 at 8:29:30 PM UTC-4 Glen Newton wrote: > Hello, > > Thanks for this, and also sorry for the lazy question. > > My testing on my machine indicates a limit of 2GB total limit on compile: > > too much data in section SXCOFFTOC (over 2e+09 bytes) > too much data in section SDATA (over 2e+09 bytes) > > I have built a bash script that creates large random files of a fixed size > and then generates a short Go program that embeds them, increasing the > number of files with each run, so you can see when your compile fails. > You can find the bash and instructions here: > https://github.com/gnewton/test_go_embed > > I just threw it together, so might be a little rough around the edges. > Feedback welcome. > > Question: Wondering why the above error complains about too much data in > *two* sections: are the embedded files stored across multiple sections (I > know nothing of how this is done internally to the elf format)? > > Thanks, > Glen > > On Monday, September 20, 2021 at 5:11:45 PM UTC-4 Ian Lance Taylor wrote: > >> On Mon, Sep 20, 2021 at 2:04 PM Glen Newton <glen....@gmail.com> wrote: >> > >> > I am testing this with an increasing number of 1GB files. At 3 files, >> and I am getting this error: >> > >> > compile: writing output: write $WORK/b001/_pkg_.a: no space left on >> device >> > >> > I haven't been able to find how to change $WORK to point to a larger >> partition. Suggestions? >> >> Set the TMPDIR environment variable to control temporary files in >> general, or the GOTMPDIR environment variable to control just >> temporary files created by the go tool. >> >> Ian >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/85284731-937b-4253-a6b5-f345d7f5ccc4n%40googlegroups.com.
