Hi Wael, I am curious if the approach outlined in this branch of the thread has continued to work for you so far, vs. perhaps you hit a snag after the initial success you reported below?
Regards, thepudds On Wednesday, March 13, 2019 at 2:46:59 PM UTC-4, Wael Nasreddine wrote: > > On Monday, March 11, 2019 at 12:33:29 PM UTC-7, thepud...@gmail.com wrote: >> >> Hi Wael, >> >> Sorry, I am not quite following what you have and have not tried yet, and >> which issues you have hit with which techniques. >> > > No worries, I'll try to address your questions to clarify the issues I'm > having. > > >> >> Is one of the issues that the 'vcs' cache directory changes, even if the >> actual code you need for your build has not changed? >> > > I also noticed the download directory changing even if the actual code > changes, but I might be incorrect on this assumption as I could not > validate it today. > > >> >> If so, I wonder if you might be able use a filesystem-based module cache >> via 'GOPROXY=file:///file/path', which could avoid using the 'vcs' >> directory at build time? >> >> It sounds like you have looked into multiple options at this point, so >> this might be well known to you at this point, but: >> >> * The module download cache location is controlled by GOPATH. In >> particular, 'go mod download', 'go build', etc. populate the module cache >> in GOPATH/pkg/mod. >> * In addition, when you want to use a particular module cache, you can >> tell the 'go' command to use a local module cache by setting >> 'GOPROXY=file:///file/path' environment variable. >> * You can put those two things together: >> >> # Populate a module download cache in /tmp/gopath-for-cache >> $ GOPATH=/tmp/gopath-for-cache go mod download >> >> # Build using the contents of the module download cache in >> /tmp/gopath-for-cache >> $ GOPROXY=file:///tmp/gopath-for-cache/pkg/mod/cache/download go >> build >> >> Note that /tmp/gopath-for-cache/pkg/mod/cache/download would not contain >> the 'vcs' directory. >> > > Using GOPROXY worked, at least I'm not seeing any network attempt. I just > must validate that the contents of the download do not change unless the > actual code of the module has changed. > > >> >> I understand those are not the exact steps you would follow, but perhaps >> something along those lines could be adapted within your constraints? >> >> Also, note that even though you are setting the GOPROXY environment >> variable in the steps above, there is no actual proxy process involved, and >> everything is just being read directly from the local filesystem. An even >> more detailed example is here in this "Go Modules by Example" walk-through: >> https://github.com/go-modules-by-example/index/tree/master/012_modvendor >> >> Sorry if anything here is off-base, or if this is not helpful. >> >> Regards, >> thepudds >> >> On Monday, March 11, 2019 at 1:01:04 PM UTC-4, Manlio Perillo wrote: >>> >>> On Monday, March 11, 2019 at 4:22:04 PM UTC+1, Wael Nasreddine wrote: >>> >>> > [...] >>> >>> >>>> On Sunday, March 10, 2019 at 6:59:07 PM UTC-7, Manlio Perillo wrote: >>>>>>> >>>>>>> On Monday, March 11, 2019 at 12:30:02 AM UTC+1, Wael Nasreddine >>>>>>> wrote: >>>>>>>> >>>>>>>> TL;DR Given a Go module, assuming that I have already done `go mod >>>>>>>> download`: Is it possible to prevent network access if I delete the >>>>>>>> entire >>>>>>>> `$GOPATH/pkg/mod/cache`? >>>>>>>> >>>>>>>> >>>>>>> Another solution is to delete only $GOPATH/pkg/mod/cache/vcs. This >>>>>>> is the directory that takes more disk space since it contains the full >>>>>>> history of the vcs. >>>>>>> >>>>>> >>>>>> That was my first attempt, however it did not work as expect because >>>>>> the contents of $GOPATH/pkg/mod/cache/download does change anytime a new >>>>>> version is created upstream. My hash failed to match multiple times a >>>>>> day! >>>>>> >>>>> >>>>> It may be caused by the missing pkg/mod/cache/vcs, but it seems >>>>> unusual. >>>>> I was assuming GOPATH was on a temporary directory. >>>>> And indeed you wrote that GOPATH is set to $NIX_BUILD_TOP/go, but >>>>> later you wrote that it is set to a temporary directory. >>>>> >>>>> >>>> I am removing the entire cache folder, as the content changes when the >>>> git repository of any of the dependencies change. So if I would hash the >>>> cache folder to A and one of the dependencies get a pull request merged >>>> upstream, even though it does not technically change the version that Go >>>> is >>>> building with you will still get B when you hash it. This break the >>>> concept >>>> of packaging :( >>>> >>>> >>> Do you perhaps have the same requirements as in the thread >>> https://groups.google.com/forum/#!topic/golang-dev/DD88cds-LuI >>> as reported by Nicolas Mailhot? >>> >>> That is, you need to patch the upstream source but keep the same >>> version, because you can't (or don't want to) update all the versions of >>> the required modules. >>> >>> In this case vendoring is, IMHO, currently the only solution, because >>> the go tool ignores the go.sum files in this case. >>> You can try to open a bug report about the incorrect behavior of `go mod >>> vendor` when using cgo. >>> >>> Or you can patch cmd/go. However instead of writing a patch specific to >>> Nix, I suggest to write a more generic patch. >>> >>> You can add a new module download mode, e.g. `-mod trust` to instruct >>> cmd/go to not check go.sum. >>> The future notary can be disabled with GONOVERIFY, so its not a problem. >>> Maybe you can use GONOVERIFY to decide if the checksum should be ignored. >>> >>> Finally, you can try to coordinate with other package managers, since >>> this seems to be a shared problem. >>> >>> Here is a patch. It seems to work but one the Go test fails, and it >>> should probably be updated: >>> https://gist.github.com/perillo/813b52dcff8824da3a2859828961d08b >>> >>> An alternative is to check for trust mode in the initGoSum function: >>> https://gist.github.com/perillo/1b3d3216c59f689e52ee7357d2a99136 >>> >>> Finally the last alternative is to check for trust mode in the >>> checkGoMod function, but this will also disable the notary. Maybe it is >>> the correct thing to do: >>> https://gist.github.com/perillo/59f3be87dd2b64d7d734c74d1f344cd5 >>> >>> > [...] >>> >>> >>> Manlio Perillo >>> >> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
