Thanks for feedback, comments below On Thu, Mar 18, 2021 at 3:35 PM Jay Conrod <jaycon...@google.com> wrote:
> > Installing a command within the context of the current module is more > complicated. I'd suggest this: > > # Add a dependency on the tool (only need to do this once) > go get -d github.com/google/go-licenses > > # Import the command from tools.go > > # See below > > > # Sync vendor directory (only if vendoring; only after changing go.mod > with 'go get' above) > go mod vendor > > # Install the command > go install github.com/google/go-licenses > > # Alternatively, build and write to some other directory > go build -o bin/go-licenses github.com/google/go-licenses > > I bent this a bit, and it seems to work: 1) add tools.go and import the tool 2) go mod vendor 3) go build IOW, I skipped `go get -d` and it all seems OK. Now it builds. If I run `go mod tidy` it removes a line from my go.mod (` github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b // indirect`) and then refuses to build: ``` go: inconsistent vendoring in /src: github.com/golang/glog@v0.0.0-20160126235308-23def4e6c14b: is marked as explicit in vendor/modules.txt, but not explicitly required in go.mod run 'go mod vendor' to sync, or use -mod=mod or -mod=readonly to ignore the vendor directory ``` Now I run `go mod tidy` again and it adds a line to `vendor/modules.txt` and happily compiles again. Subsequent runs of either command seem to be no-op at this point. Is it by design that these two commands are unstable like that? Now that it builds, extra fun. This tool in particular seems to try to read a DB file from a subdir of the directory its source code came from (as per `runtime.Caller(0)`) which ...doesn't work at all with vendoring, since the dir doesn't contain Go code. AFAIK there's not a way to ask `go mod vendor` to include the whole repo is there? I can't find a spec for modules.txt - it would be super cool if I could add some tag in there... As it stands, all this was for naught. > *2. "writing go.mod cache" error messages* > > This error message should be a lot better. Sorry about that. > I mean, "mkdir /home/thockin/go: not a directory" is pretty concise - it's not a directory (on purpose) because I really don't want Go randomly deciding to write stuff to my homedir (which I suspected was happening, and that's why I made it a non-directory :) > *3. 'go get' updating go.mod and go.sum* > > This is an unfortunate consequence of 'go get' being overloaded for > downloading, installing, and managing package versions. Ideally, we want to > separate these roles into different commands: 'go get' should be used to > manage dependencies (changing requirements in go.mod and go.sum), and 'go > install' should be used to install commands. > > We've taken a couple steps toward this in 1.16. The 'go install cmd@version' > form is new. Also, 'go install', 'go build', and other commands no longer > change go.mod or go.sum automatically when something is missing. > >From where I site, I like the distinction, and I might even argue for explicitness. E.g. if I said `go get -mod=vendor` I feel like I am unambiguously asking the tool to fetch the code and put it in ./vendor and update the module files. If I said just `go get` I (not so unambiguously) mean "global" (for me). I find the automatic-detection of module mode very confusing, personally. Tim On Thu, Mar 18, 2021 at 3:36 PM 'Tim Hockin' via golang-nuts < > golang-nuts@googlegroups.com> wrote: > >> First: go version go1.16 linux/amd64 >> >> In one of my side-projects, I build a container image. It turns out >> to be useful to people, but I got a request to put the license files >> for all code used in the binary into the container. Lawyers, what can >> you do? >> >> So these people sent me a PR to gather the licenses with `go install >> github.com/google/go-licenses` <http://github.com/google/go-licenses> in >> the `make container` rule. Here's >> the friction. >> >> 1) Their PR includes dozens of new lines in go.sum, which makes me >> grumpy. I set out to see if I can do better. >> >> 2) When I run that command on my workstation I get: >> >> ``` >> $ go install github.com/google/go-licenses >> cannot find package "." in: >> /home/thockin/src/go/src/ >> k8s.io/git-sync/vendor/github.com/google/go-licenses >> ``` >> >> I don't know what to make of that message. It's not in vendor/ yet - >> I want to put it there (I think?). >> >> So I am left wondering - what is the normal flow for "I want to use >> this Go tool in my build" ? >> >> I tried `-mod=mod`: >> >> ``` >> $ (unset GOPATH; go install -mod=mod github.com/google/go-licenses) >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: finding module for package github.com/google/go-licenses >> go: downloading github.com/google/go-licenses >> v0.0.0-20201026145851-73411c8fa237 >> mkdir /home/thockin/go: not a directory >> ``` >> >> I keep hearing that GOPATH is dead (and frankly good riddance) but >> this tries to write to GOPATH or ~/go (which is also annoying). I >> don't want this project's build spilling over into my homedir. >> >> I tried `go get` but it's the same. >> >> ``` >> $ (unset GOPATH; go get github.com/google/go-licenses) >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: writing go.mod cache: mkdir /home/thockin/go: not a directory >> go: downloading github.com/google/go-licenses >> v0.0.0-20201026145851-73411c8fa237 >> go get github.com/google/go-licenses: mkdir /home/thockin/go: not a >> directory >> ``` >> >> What do I *want*? I want to download a tool repo, build it, drop the >> binary into ./bin/tools (or something) and then either vendor it or >> have no artifacts. >> >> Since GOPATH seems to be a requirement, I thought maybe I can fake it: >> >> ``` >> $ (export GOPATH="`pwd`/.gopath"; export GOBIN=`pwd`/bin/tools; mkdir >> -p "$GOBIN"; go get github.com/google/go-licenses) >> go: downloading github.com/google/go-licenses >> v0.0.0-20201026145851-73411c8fa237 >> go: downloading github.com/otiai10/copy v1.2.0 >> go: downloading github.com/spf13/cobra v0.0.5 >> go: downloading github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b >> go: downloading github.com/google/licenseclassifier >> v0.0.0-20190926221455-842c0d70d702 >> go: downloading golang.org/x/tools v0.0.0-20191118222007-07fc4c7f2b98 >> go: downloading gopkg.in/src-d/go-git.v4 v4.13.1 >> go: downloading github.com/spf13/pflag v1.0.5 >> go: downloading github.com/inconshreveable/mousetrap v1.0.0 >> go: downloading github.com/sergi/go-diff v1.0.0 >> go: downloading gopkg.in/src-d/go-billy.v4 v4.3.2 >> go: downloading golang.org/x/crypto v0.0.0-20191117063200-497ca9f6d64f >> go: downloading golang.org/x/sys v0.0.0-20191119060738-e882bf8e40c2 >> go: downloading github.com/emirpasic/gods v1.12.0 >> go: downloading github.com/jbenet/go-context >> v0.0.0-20150711004518-d14ea06fba99 >> go: downloading github.com/src-d/gcfg v1.4.0 >> go: downloading github.com/kevinburke/ssh_config >> v0.0.0-20190725054713-01f96b0aa0cd >> go: downloading github.com/mitchellh/go-homedir v1.1.0 >> go: downloading github.com/xanzy/ssh-agent v0.2.1 >> go: downloading golang.org/x/net v0.0.0-20191119073136-fc4aabc6c914 >> go: downloading gopkg.in/warnings.v0 v0.1.2 >> go get: added github.com/google/go-licenses >> v0.0.0-20201026145851-73411c8fa237 >> ``` >> >> This does actually build my binary (and leaves me one more thing to >> `make clean`), but it also writes to go.mod and go.sum. I thought I'd >> complete the task and vendor the code. `go mod vendor` adds a line to >> vendor/modules.txt: >> >> ``` >> # github.com/google/go-licenses v0.0.0-20201026145851-73411c8fa237 >> ``` >> >> ...but doesn't actually vendor the code. OK. What if I just clean >> up? `go mod tidy`. This leaves that vendor/modules.txt change but >> removes almost all of the other changes to go.sum. Curiously, it >> leaves some: >> >> ``` >> $ git diff >> diff --git a/go.mod b/go.mod >> index ced314c..cfe7bca 100644 >> --- a/go.mod >> +++ b/go.mod >> @@ -5,6 +5,7 @@ require ( >> github.com/go-logr/logr v0.1.0 // indirect >> github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b // >> indirect >> github.com/prometheus/client_golang v0.9.2 >> + golang.org/x/sync v0.0.0-20190423024810-112230192c58 // indirect >> ) >> >> go 1.15 >> diff --git a/go.sum b/go.sum >> index b9f0527..4e159c8 100644 >> --- a/go.sum >> +++ b/go.sum >> @@ -19,5 +19,6 @@ github.com/prometheus/common >> v0.0.0-20181126121408-4724e9255275/go.mod h1:daVV7q >> github.com/prometheus/procfs v0.0.0-20181204211112-1dc9a6cbc91a >> h1:9a8MnZMP0X2nLJdBg+pBmGgkJlSaKC2KaQmTCk1XDtE= >> github.com/prometheus/procfs >> v0.0.0-20181204211112-1dc9a6cbc91a/go.mod >> h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= >> golang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod >> h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= >> -golang.org/x/sync v0.0.0-20181108010431-42b317875d0f >> h1:Bl/8QSvNqXvPGPGXa2z5xUTmV7VDcZyvRZ+QQXkXTZQ= >> golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod >> h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= >> +golang.org/x/sync v0.0.0-20190423024810-112230192c58 >> h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU= >> +golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod >> h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= >> ``` >> >> If I reset the whole repo and run `go mod tidy` it does not make this >> change. >> >> Googling around I found a suggestion to use a different modfile. That >> sounds like a hack, but I'll try it. >> >> I made a new, nearly-empty `go.tools.mod`: >> >> ``` >> $ (export GOPATH="`pwd`/.gopath"; export GOBIN=`pwd`/bin/tools; mkdir >> -p "$GOBIN"; go get -modfile=go.tools.mod >> github.com/google/go-licenses) >> ``` >> >> This works and wrote to go.tools.mod a created a new go.tools.sum. I >> still can't figure how to vendor the tool. I guess I can just >> .gitignore those, but this is starting to feel like I made a wrong >> turn at Albuquerque. >> >> There has to be a "normal" path here, right? I feel like I shouldn't >> have to fight so hard to make a more self-contained build. >> >> Can someone clue me in? >> >> Tim >> >> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/CAO_Rewb_5vBW4fuEa6i-XLLRiHqgEibxXTce3hHmOHfgAsRpfQ%40mail.gmail.com >> . >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAO_RewbogKj2-jjEs3AHiCGCE1twKt-y_uU%2BnhuEZHm82eoqtA%40mail.gmail.com.
