Reading Alekseys description, it does seem to be making a bit more sense. The C/C++ compilers use a "file" as a compilation unit. A file is converted to an object file which must contain all of its dependencies. So the includes will need to copy all of the code that they are importing into the file that is being compiled.
In Golang, the object file is more of a blackbox which contains only the necessary data that is needed. I'm assuming that "necessary" relates to type checking and symbol resolution mostly. It seems that one key difference is that Golang uses a package as a compilation unit, while C++ uses a file. If Golang used a file also and not a module structure, then it seems that similar issues or a significant decrease in performance would be observed. You would have more object files for one and there would be more dependencies between files. On Saturday, 14 November 2020 at 05:57:27 UTC aleksey...@gmail.com wrote: > There is no direct relationship between headers and object files in C > or C++. Compilation process is two stage: > > 1. Source files are compiled into object files > 2. Object files are linked together into executable of sorts > > (Actually it's a three stage process, but i'm going to describe it > using two stages). > > Stages are isolated from each other and to some extent autonomous. > Object files are kind of intermediate representation of source code > that is later used to produce machine code. I'm sure someone can > correct me on this, but for the sake of simplicity, i think it's OK to > think of them as of IR. > > When *compilation unit* is compiled into an object file, it is also > separated from other units. To compile it separately from other units > all relevant source code has to be pulled into the current unit and > compiled. So `#include <something.h>` doesn't include just > something.h, it includes something.h, then all includes that > something.h includes and so on. This is a process similar to > amalgamation of source code, everything is copied into one place and > then compiled as a single unit. After all units are compiled, they > might be joined together by a linker either into a static library, > dynamic library or executable. > > This is actually more sophisticated than that, but it does allow you > to do some cool stuff like you can compile your source code into > objects, then ship object files and then link them elsewhere. In fact, > static libraries are just a bunch of object files packed together, but > headers are still required because you need symbol names to refer to > on source code level, therefore libraries are shipped with headers: > you compile with headers and then link with objects. > > Since it's the separate stages, you could, for instance, write your > own headers for 3rd party objects, think open source headers for > closed source DirectX SDK. > > This description is very superficial and doesn't cover a lot of what > is really going on. The process is very flexible and allows to do all > kinds of stuff in various combinations. Alas this process is also not > very fast and requires some costly steps like you need to pull all > required source code into a single unit to compile it. > > Modern C++ is also using a lot of templates, even if you're not > writing templates, you're going to use templates from the standard > library and to use templates you need to transform (instantiate) each > template into concrete code and then (simply put) compile instantiated > template as regular non-templated source code. Because every > compilation unit is being "amalgamated", this process has to be > repeated for every unit, which also takes some time. > > There is such thing as C++ modules, but they are quite new > (standardized like a month ago) and not yet widespread. I think they > should be more similar to Go *packages* when source code files are > logically joined into a single entity and for that entity another > intermediate representation is created which is called BMI (binary > module interface) even though it doesn't have to be binary, so > sometimes it's called CMI (compiler module interface). > > This CMI is basically a compiler cache, a package, or in terms of C++, > a module interface, can be compiled once and then reused to compile > object files without recompiling the same source code for every unit. > > Regarding how packages compilation actually works in Go - this is an > interesting topic. I'm afraid i won't be able to explain it more or > less correctly and i would be glad to read about it too. > > сб, 14 нояб. 2020 г. в 04:17, kev kev <kevthem...@gmail.com>: > > > > > > Thanks for the answer. If C/C++ has object files, is it not possible to > see “something.h” and then fetch the corresponding object file? > > > > With go, if I import “package something” and that package imports > another package called “package bar” then at some point I will need to > compile “bar” and “something”. This to me is like your header example. > > > > I think you are maybe saying that this traversal is only done once for > golang and the information is stored in an object file? While in C, the > header traversal is done each time I see include? > > On Saturday, 14 November 2020 at 00:14:41 UTC Kevin Chowski wrote: > >> > >> C/C++ also has object file caching (depending on how your build is set > up, I guess). In C/C++ the issue is that you need to possibly open a large > number of header files when you import any header file. > >> > >> For example, if I write a file "main.c" which imports "something.h", > which in turn imports "another.h" and "big.h", and compile just main.c, the > compiler has to open all three header files and include them in the parsing > of main.c in order for the compilation to correctly move forward. In Go, > the compiler arranges things such that it only has to open one file per > package that is imported. The post you linked goes into greater detail, so > I will avoid duplicating the details for now, but feel free to ask a more > specific question and I can try to answer. > >> > >> There's a bit of nuance there, which the post also goes into: Go's > strategy ends up requiring that some package much be compiled before any > package which imports it is compiled. In C/C++ the ordering is a little > more flexible due to the more decoupled nature of header files, meaning > that theoretically more builds could occur in parallel. But I suspect that > in your average Go program the dependency tree would still allow you to > execute a large number of builds in parallel. > >> > >> Also note that the article claims this is "the single biggest reason" > Go compilation is fast, not the only one. There are lots of smaller, yet > important, reasons as well. For example, parsing the language is pretty > straightforward because it is not very complex, and linking the final > binary together is continually being optimized. Plus there are no > turing-complete meta-language features like the templates C++ compilers > have to deal with ;) > >> > >> As for your following, the whole set of files in some package are the > compilation unit, at least as far as I understand the terms. This is > because if a.go and b.go are both in the same package (e.g. in the same > directory), code in a.go can call code in b.go without explicitly declaring > anything. So before the code in a.go can be fully compiled into an object > file, b.go must be considered as well. > >> On Friday, November 13, 2020 at 3:54:34 PM UTC-7 kev kev wrote: > >>> > >>> I recently read the post by Rob Pike about language choices for > Golang: https://talks.golang.org/2012/splash.article#TOC_5. > >>> > >>> The seventh point refers to how Golang handles dependencies. It > mentions an "object file" for packages that a _dependent_ reads. > >>> > >>> Below I go through my interpretation of this section: > >>> > >>> Example: > >>> > >>> package A imports package B. > >>> > >>> When I compile package A, package B would have already been compiled. > What package A receives is not the AST of package B, but an "Object file". > This object file only reveals data about the publicly accessible symbols in > that package. From the example, if B had a private struct defined inside of > it, this private struct would not be in the object file. > >>> > >>> This part seems to make sense for me, hopefully I did not make any > mistakes. > >>> > >>> It seems that the speedup compared to C/C++ is because the object file > is created once per package, while in C/C++ you need to re-compile the > thing you are including each time? > >>> > >>> Followup question: > >>> > >>> Is a single file a compilation unit or is it a package? > >>> > >>> Thanks > >>> > > -- > > You received this message because you are subscribed to the Google > Groups "golang-nuts" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to golang-nuts...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/2e237e13-37c9-4741-8ea1-67f813923fafn%40googlegroups.com > . > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/1f55ed1f-de3f-4950-8b1f-c81feaaa8772n%40googlegroups.com.
