On a second reading, it turns out I do not proof-read all of my emails
thoroughly enough before hitting send. My apologies for any confusion
and/or alarm.
In the meantime, I have solved the problem, so I'm following up here with
more context and my solution in case anyone else wants to do something
similar and might find the information useful. I also encountered what may
be a small documentation hole (or maybe I'm missing something).
The GitHub Vet project reads Go repositories from GitHub and performs
static analysis. As part of improving the analyzers, I've begun
incorporating the type-checker.
What I've seen work so far is a setup like this, based on the packages
package
var fset token.FileSet
config := packages.Config{
Dir: srcDir,
Fset: &fset,
Mode: packages.NeedTypesInfo | packages.NeedSyntax | packages.NeedTypes,
}
pkgs, err := packages.Load(&config, "./...")
Unfortunately, this approach requires that I first write the repository
contents to disk, so they can be read in from srcDir. I do not want to send
the repository contents to disk if I can avoid it.
But it seems from the packages.Config
<https://pkg.go.dev/golang.org/x/tools/go/packages#Config> documentation
that using the disk *may* not be necessary. packges.Config contains this
very suggestive Overlay field which I have not been able to figure out how
to use (documentation below).
// Overlay provides a mapping of absolute file paths to file contents.
// If the file with the given path already exists, the parser will use the
// alternative file contents provided by the map.
//
// Overlays provide incomplete support for when a given file doesn't
// already exist on disk. See the package doc above for more details.
Overlay map[string][]byte
(FWIW, I couldn't find the details referenced in the package doc).
I have tried providing a map[string][]byte to this argument with no
success. I am using the path of each .go file as a key, and including the
[]byte contents of the file. Based on the what I am seeing, ISTM that there
is an undocumented dependency between the paths used as keys in the Overlay
and the Config.Dir field which I was unable to get a handle on using
black-box hackery.
I also tried digging into the source of the packages.Load method, but
there's a layer of indirection involved via a third-party driver so I found
it challenging to get a handle on where Overlay is actually being used.
So I'm left wondering whether there's any way to make the packages package
work without relying on the filesystem. I'm sure there is, but it's
difficult (for me at least) to understand.
OTOH I *was* able to get the loader package to type-check from memory by
parsing the files into a map[string][]*ast.File using something like the
below snippet.
var filesByPath map[string][]*ast.File
config := loader.Config{
Fset: fset,
AllowErrors: true,
TypeChecker: types.Config{
Importer: importer.ForCompiler(fset, "gc", nil)
},
}
for dir, files := range filesByPath {
config.CreateFromFiles(dir, files...)
}
prog, err := config.Load()
So here's hoping this is helpful to others who may have the same (albeit
obscure) goal. I'm aware the loader package predates Go modules, so this
approach may miss something important for some use-cases. For my purposes,
I want to avoid downloading / type-check module dependencies anyway.
Sincerely,
K. Alex Mills
On Wed, Jan 20, 2021 at 8:03 PM K. Alex Mills <k.alex.mi...@gmail.com>
wrote:
> Hi All,
>
> Right now, I can run the type-checker by writing the GitHub tarball
> contents to disk and using the packages.Load function to parse and
> type-check the contents.
>
> Unfortunately, all I've been able to make work at the moment with
> packages.Load would entail writing the entire repository contents to
> disk... only to read them back into memory... and then delete them. This
> works, but it could be better.
>
> I'm curious if anyone is aware of a way to parse and type-check multiple
> packages (and making type information available to multiple packages in the
> process) for source code that "lives" in memory.
>
> I've tried using the very conspicuous Overlay field from packages.Config
> and passing a map from a fake filepath to the contents of the file in
> memory. I've found it challenging to find an acceptable combination of Dir
> and Overlay that will make this work (if one exists).
>
> I've also noticed that packages provides this option for external drivers
> but I'm not sure it fits my use-case, since it seems like it would involve
> IPC.
>
> If I'm passionate about not writing these files to disk, is there a way I
> can invoke the type-checker for multiple packages without reinventing a
> large part of what packages provides?
>
> Thanks,
>
> K. Alex Mills
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/CALJzkY_2GOwTd%3DmUeqJ0o-xkz7eUVgBaHcz5RRchr0TWWQSSEA%40mail.gmail.com.
