Hmm, I'm not sure what's going on. I ran it again four times in a row
and I got better results, but the timings show that there is a lot fo
difference between the slow and fast cases (no idea why)
$ time ./mach build binaries
0:00.81 /usr/bin/make -j8 -s binaries
0:03.90 From ./dist/public: Kept 0 existing; Added/updated 0; Removed
0 files and 0 directories.
0:03.90 From ./dist/sdk: Kept 0 existing; Added/updated 0; Removed 0
files and 0 directories.
0:03.95 From ./dist/private: Kept 0 existing; Added/updated 0; Removed
0 files and 0 directories.
0:04.02 From ./dist/idl: Kept 1155 existing; Added/updated 0; Removed
0 files and 0 directories.
0:04.13 From ./dist/bin: Kept 0 existing; Added/updated 0; Removed 0
files and 0 directories.
0:04.32 From ./dist/include: Kept 3519 existing; Added/updated 0;
Removed 0 files and 0 directories.
0:06.42 From _tests: Kept 11158 existing; Added/updated 0; Removed 0
files and 0 directories.
Your build was successful!
real 0m13.277s
user 0m9.969s
sys 0m2.929s
ehsanakhgari sparky ~ moz mozilla-central include $ time ./mach
build binaries
0:00.25 /usr/bin/make -j8 -s binaries
0:01.49 From ./dist/public: Kept 0 existing; Added/updated 0; Removed
0 files and 0 directories.
0:01.49 From ./dist/sdk: Kept 0 existing; Added/updated 0; Removed 0
files and 0 directories.
0:01.50 From ./dist/private: Kept 0 existing; Added/updated 0; Removed
0 files and 0 directories.
0:01.64 From ./dist/idl: Kept 1155 existing; Added/updated 0; Removed
0 files and 0 directories.
0:01.67 From ./dist/bin: Kept 0 existing; Added/updated 0; Removed 0
files and 0 directories.
0:01.89 From ./dist/include: Kept 3519 existing; Added/updated 0;
Removed 0 files and 0 directories.
0:03.36 From _tests: Kept 11158 existing; Added/updated 0; Removed 0
files and 0 directories.
Your build was successful!
real 0m10.459s
user 0m11.120s
sys 0m1.914s
ehsanakhgari sparky ~ moz mozilla-central include $ time ./mach
build binaries
0:00.26 /usr/bin/make -j8 -s binaries
0:01.53 From ./dist/public: Kept 0 existing; Added/updated 0; Removed
0 files and 0 directories.
0:01.54 From ./dist/sdk: Kept 0 existing; Added/updated 0; Removed 0
files and 0 directories.
0:01.54 From ./dist/private: Kept 0 existing; Added/updated 0; Removed
0 files and 0 directories.
0:01.67 From ./dist/idl: Kept 1155 existing; Added/updated 0; Removed
0 files and 0 directories.
0:01.80 From ./dist/bin: Kept 0 existing; Added/updated 0; Removed 0
files and 0 directories.
0:01.93 From ./dist/include: Kept 3519 existing; Added/updated 0;
Removed 0 files and 0 directories.
0:03.37 From _tests: Kept 11158 existing; Added/updated 0; Removed 0
files and 0 directories.
Your build was successful!
real 0m9.334s
user 0m10.229s
sys 0m1.924s
ehsanakhgari sparky ~ moz mozilla-central include $ time ./mach
build binaries
0:00.18 /usr/bin/make -j8 -s binaries
0:04.53 From ./dist/sdk: Kept 0 existing; Added/updated 0; Removed 0
files and 0 directories.
0:04.54 From ./dist/private: Kept 0 existing; Added/updated 0; Removed
0 files and 0 directories.
0:04.54 From ./dist/public: Kept 0 existing; Added/updated 0; Removed
0 files and 0 directories.
0:04.65 From ./dist/idl: Kept 1155 existing; Added/updated 0; Removed
0 files and 0 directories.
0:04.67 From ./dist/bin: Kept 0 existing; Added/updated 0; Removed 0
files and 0 directories.
0:04.83 From ./dist/include: Kept 3519 existing; Added/updated 0;
Removed 0 files and 0 directories.
0:06.10 From _tests: Kept 11158 existing; Added/updated 0; Removed 0
files and 0 directories.
Your build was successful!
real 0m14.923s
user 0m9.508s
sys 0m1.580s
On 2013-10-02 12:20 PM, Gregory Szorc wrote:
8.8s here!
~1.5 is startup and checking the build backend is up to date (lots of
stats)
~1.5s is processing install manifests.
Rest is make processing.
The fact that your machine spent ~20s doing install manifest processing
tells me:
a) Your directory tree wasn't cached (try running again)
b) Your I/O layer is slow (I'm building with an SSD)
c) Something else wonky creating slow I/O
0:00.14 /usr/bin/make -j8 binaries
0:00.53 MOZBUILD_BACKEND_CHECKED= /usr/bin/make -C js/src
backend.RecursiveMakeBackend.built
0:00.82 make[1]: `backend.RecursiveMakeBackend.built' is up to date.
0:01.16 MOZBUILD_BACKEND_CHECKED= /usr/bin/make -C js/src
backend.RecursiveMakeBackend.built
0:01.45 make[2]: `backend.RecursiveMakeBackend.built' is up to date.
0:01.56 From ./dist/public: Kept 0 existing; Added/updated 0; Removed
0 files and 0 directories.
0:01.56 From ./dist/sdk: Kept 0 existing; Added/updated 0; Removed 0
files and 0 directories.
0:01.57 From ./dist/private: Kept 0 existing; Added/updated 0; Removed
0 files and 0 directories.
0:01.68 From ./dist/idl: Kept 1155 existing; Added/updated 0; Removed
0 files and 0 directories.
0:01.72 From ./dist/bin: Kept 0 existing; Added/updated 0; Removed 0
files and 0 directories.
0:01.86 From ./dist/include: Kept 3518 existing; Added/updated 0;
Removed 0 files and 0 directories.
0:03.10 From _tests: Kept 11158 existing; Added/updated 0; Removed 0
files and 0 directories.
0:03.10 /usr/bin/make recurse_binaries
On 10/2/13 5:42 PM, Ehsan Akhgari wrote:
I just did a no-op ./mach build binaries on my debug build on a Mac, and
it took about 28 seconds.
$ time ./mach build binaries
0:01.96 /usr/bin/make -j8 -s binaries
0:12.19 From ./dist/public: Kept 0 existing; Added/updated 0; Removed
0 files and 0 directories.
0:12.22 From ./dist/sdk: Kept 0 existing; Added/updated 0; Removed 0
files and 0 directories.
0:12.38 From ./dist/private: Kept 0 existing; Added/updated 0; Removed
0 files and 0 directories.
0:12.70 From ./dist/bin: Kept 0 existing; Added/updated 0; Removed 0
files and 0 directories.
0:13.09 From ./dist/idl: Kept 1155 existing; Added/updated 0; Removed
0 files and 0 directories.
0:13.11 From ./dist/include: Kept 3519 existing; Added/updated 0;
Removed 0 files and 0 directories.
0:20.46 From _tests: Kept 11158 existing; Added/updated 0; Removed 0
files and 0 directories.
Your build was successful!
real 0m28.291s
user 0m8.685s
sys 0m4.178s
Do you have any plans to bring down the overhead? Do you know where
this overhead is coming from?
Thanks!
Ehsan
On 2013-10-01 9:17 PM, Mike Hommey wrote:
Hi,
If you've read the "You want faster builds, don't you" thread, you may
know that some build improvements have recently landed.
I just landed the most important part of it all, and we should now be in
a much better place, but, as I'm very cautious, and as this is
incremental improvements to an existing complex build system that is
hard to improve all at once without some subtle breakages, this is
opt-in. It also doesn't work with pymake because of bug 918652.
At this point, you probably want to know what it is and how to use it.
There is now a new target for incremental C/C++ rebuilds. What this
means
is, you build once like usual. Then after you do your C/C++ changes,
instead of:
- mach build or make -C objdir, which takes forever
- mach build subdirectory/of/the/changes, which sometimes rebuilds
toolkit/library, sometimes not, depending what you're rebuilding.
- make -C objdir/subdirectory/of/the/changes && make -C
objdir/toolkit/library, which may actually not be enough.
you can now do:
- mach build binaries
or
- make -C objdir binaries
It will rebuild your changes and everything that needs rebuilding
because
of them. It will also do that quickly.
There are a few caveats:
- it only handles C/C++ changes, including headers. It doesn't handle js
modules, chrome data, etc.
- it does *not* handle changes to xpidl, webidl, ipdl. yet. There's a
followup for this to happen: bug 921309.
- it doesn't handle changes to nss, nspr, icu or ffi. If you do changes
there, you still need to run a normal build.
- it doesn't work without doing a normal build first.
- while it shouldn't break your builds, it might subtly skip what you
would expect it to build. If it does, please file a bug or contact me
on irc. You can still use the old ways until your issues are fixed.
Something else that I landed today is support to skip directories during
a normal build when they're not relevant to the build. As always, I'm
overcautious and this is opt-in. If you want to opt-in for this (and
future experimental improvements), please add "export
MOZ_PSEUDO_DERECURSE=1" to your mozconfig. Except if you're using
pymake, sadly. The more people test those experimental improvements, the
quicker they can become the default for everyone.
For those interested in the gory details, I'll post some on my blog
within
the next few days.
Mike
_______________________________________________
dev-builds mailing list
dev-bui...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-builds
_______________________________________________
dev-builds mailing list
dev-bui...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-builds
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
