You could try building the universe scope for ast.NewPackage from types.Universe. For example
https://play.golang.org/p/1E5Iu4vW3g9 func NewPackage(fset *token.FileSet, files map[string]*ast.File) (*ast.Package, error) { univ, err := universe() if err != nil { return nil, err } return ast.NewPackage(fset, files, dummyImporter, univ) } func dummyImporter(imports map[string]*ast.Object, importPath string) (*ast.Object, error) { pkg := imports[importPath] if pkg == nil { pkg = ast.NewObj(ast.Pkg, path.Base(importPath)) pkg.Data = ast.NewScope(nil) imports[importPath] = pkg } return pkg, nil } func universe() (*ast.Scope, error) { u := ast.NewScope(nil) for _, name := range types.Universe.Names() { o := types.Universe.Lookup(name) if o == nil { return nil, fmt.Errorf("failed to lookup %s in universe scope", name) } var objKind ast.ObjKind switch o.(type) { case *types.Const, *types.Nil: objKind = ast.Con case *types.TypeName: objKind = ast.Typ case *types.Builtin: objKind = ast.Fun default: return nil, fmt.Errorf("unexpected builtin %s of type %T", o.Name(), o) } obj := ast.NewObj(objKind, name) if u.Insert(obj) != nil { return nil, fmt.Errorf("types internal error: double declaration") } obj.Decl = u } return u, nil } On Saturday, 16 October 2021 at 14:38:43 UTC+1 eli...@gmail.com wrote: > On Fri, Oct 15, 2021 at 2:13 PM Steven Hartland <ste...@multiplay.co.uk> > wrote: > >> I converted my code to x/tools/go/packages >> <https://pkg.go.dev/golang.org/x/tools@v0.1.7/go/packages> and while it >> did solve the problem it's VERY slow in comparison. >> >> I have a set of 21 tests operating on a single package which has at most >> two very basic types, no imports and using go/parser >> <https://pkg.go.dev/go/parser> they take 0.011s but with go/packages >> <https://pkg.go.dev/golang.org/x/tools@v0.1.7/go/packages> that >> increases to 3.548s a 300x slow down. >> >> I'm setting a basic mode: packages.NeedName | packages.NeedSyntax >> >> The package.Load call takes ~220ms whereas ast.NewPackage only >> takes 2.7µs. >> > > Could you post a reproducer of your target package and analysis somewhere? > 220ms for packages.Load sounds like a lot. It's true that packages does a > lot more work than just the parser (*), but it's not supposed to be that > slow. In my tests a simple Load with more functionality takes 60-70ms > > (*) The type checking takes a bit of time over just parsing to AST, but > the biggest difference is loading multiple files from imports. For type > checking you need to know, when you see: > > import foo > > x := foo.Foo() > > What the type of `x` is, so go/packages has to analyze the `foo` package > as well. > > > >> >> As the resulting ast.File's are pretty much the same, I'm wondering if >> for my use case packages.Load is doing way more than I need? >> >> Another downside is for tests run in a temporary directory outside of the >> package space package.Load fails with: >> directory /tmp/tests76985775 outside available modules >> >> I fixed it by calling ioutil.TempDir with "." but that's not ideal. >> >> Thoughts? >> >> On Tue, 12 Oct 2021 at 13:42, Steven Hartland <ste...@multiplay.co.uk> >> wrote: >> >>> Thanks David, much appreciated, I will have a look at both. >>> >>> When migrating from go/ast to go/types did you hit anything of note I >>> should look out for? >>> >>> On Mon, 11 Oct 2021 at 17:06, David Finkel <david....@gmail.com> wrote: >>> >>>> >>>> >>>> On Mon, Oct 11, 2021 at 5:48 AM Steven Hartland <ste...@multiplay.co.uk> >>>> wrote: >>>> >>>>> If the ast.Files passed to ast.NewPackage includes built in types such >>>>> as int it returns an error e.g. >>>>> file1.go:5:6: undeclared name: int >>>>> >>>>> Is there a way to prevent that? >>>>> >>>> >>>> Generally, I always add the `builtin` package to the list of packages >>>> I'm parsing. >>>> I wrote a little library for exactly this kind of package loading a few >>>> years ago: >>>> https://gitlab.com/dfinkel/goastpkg/-/blob/master/go_ast_parser.go >>>> (https://pkg.go.dev/golang.spin-2.net/astpkg) >>>> >>>>> >>>>> Playground example: https://play.golang.org/p/Yg30TTzoLHP >>>>> >>>>> My goal is to take multiple files, resolve inter file dependencies >>>>> e.g. a type referencing another type in a different file and process the >>>>> resulting ast.Files. So if there is a better way to achieve this I'm all >>>>> ears. >>>>> >>>> >>>> In general, I've stopped using the `go/ast` internal references as much >>>> and have started using resolved `go/types` references as they're more >>>> reliable and better-specified. >>>> (golang.org/x/tools/go/packages >>>> <https://pkg.go.dev/golang.org/x/tools@v0.1.7/go/packages> has a >>>> LoadMode flag for generating `go/types.Info` (NeedTypesInfo >>>> <https://pkg.go.dev/golang.org/x/tools@v0.1.7/go/packages#NeedTypesInfo> >>>> )) >>>> >>>>> >>>>> Regards >>>>> Steve >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "golang-nuts" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to golang-nuts...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/golang-nuts/CAHEMsqbJoJxuo3c-mofMtzXXJhYCzV2skW2ZB3ZPY6WtA8%2BxHw%40mail.gmail.com >>>>> >>>>> <https://groups.google.com/d/msgid/golang-nuts/CAHEMsqbJoJxuo3c-mofMtzXXJhYCzV2skW2ZB3ZPY6WtA8%2BxHw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts...@googlegroups.com. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/golang-nuts/CAHEMsqYMSBUfuOUvptv6UrvBFTwFxjOhJZ5sMN-omOx5ESL5hw%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/golang-nuts/CAHEMsqYMSBUfuOUvptv6UrvBFTwFxjOhJZ5sMN-omOx5ESL5hw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/d570a7ce-a780-46d8-a323-f9c26a6c2561n%40googlegroups.com.
