I just can't figure out how to do this. Maybe it can't be done in `go list` ? Or maybe we're just missing some detail of go modules..
I have a repo (kubernetes, but minimized at https://github.com/thockin/go-list-modules) which has a "main" go module, and several other modules in the same repo. It's probable not worth explaining the history right now, but here we are. As part of the build, I need to run some code generators. They are slow, so I only want to run them if their inputs have actually changed. To do this, I need to build a list of all the individual files that are transitively depended on by the package in question. First I do a `find` for any file that has a specific comment tag, indicating that the package needs codegen. The results span several of the in-repo submodules. For each target package, I want to get the list of all deps and extract the GoFiles. Then I can use that to determine if the codegen needs to run. Where it breaks down is that I can't seem to `go list` all at once: ``` # This works within the "root" module $ go list -f '{{.GoFiles}}' ./subdir [file.go] # This does not work across modules $ go list -f '{{.GoFiles}}' ./submod/used ./submod/unused main module (example.com/m) does not contain package example.com/m/submod/used main module (example.com/m) does not contain package example.com/m/submod/unused # Nor does this work, even with module replacements $ go list -f '{{.GoFiles}}' ./staging/src/example.com/other1/used ./staging/src/example.com/other1/unused main module (example.com/m) does not contain package example.com/m/staging/src/example.com/other1/used main module (example.com/m) does not contain package example.com/m/staging/src/example.com/other1/unused ``` I can run `go list` multiple times, but that's INCREDIBLY slow - most of these submodules have common deps that are large. This re-parses everything over and over. It takes almost 60 seconds just to do `cd $dir; go list` (on the real kubernetes repo). I wrote a program against go/packages which does SLIGHTLY better, in that it finds packages in other modules but ONLY if they a) are a dep of something in the root module; and b) exist in vendor/. Note here that ./subir DOES depend on ./submod, but that is not listed. ``` $go2make2 -p ./subdir Package "example.com/other1/used" { Name: "used" PkgPath: "example.com/other1/used" GoFiles: [ "/tmp/m/go-list-modules/vendor/example.com/other1/used/file.go" ] Imports: [ ] } Package "example.com/other2/used" { Name: "used" PkgPath: "example.com/other2/used" GoFiles: [ "/tmp/m/go-list-modules/vendor/example.com/other2/used/file.go" ] Imports: [ ] } Package "example.com/m/subdir" { Name: "subdir" PkgPath: "example.com/m/subdir" Module: "example.com/m" ModuleDir: "/tmp/m/go-list-modules" GoFiles: [ "/tmp/m/go-list-modules/subdir/file.go" ] Imports: [ "example.com/m/submod/used" "example.com/other1/used" "example.com/other2/used" ] } ``` Also note that `Module` info is missing for the ones it does find. So I guess I am looking for some magical incantation that lets me process and load all of the files across multiple modules exactly once. I wrote a similar program against go/build which runs in < 3 seconds, but doesn't work properly outside of a GOPATH (not sure why). Any clues on how to approach this better would be well appreciated. Tim -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAO_RewakNiSEKk6uBFjcGbQP0AgxqT0tDcyWsY5JTz_NaZDn4A%40mail.gmail.com.
