My idea is to just skip compile (some) lisp file during
database generation stage, similar to the "$SaveParseOnly"
variable.
For the following steps, compile lisp file is important,
otherwise the repeated loading of lisp file will take more time.
See my updated patch (still a hack) for the following
benchmark result. Roughly saves 9% of build time,
not as much as I originally thought.
- Qian
SBCL with hack, 8 cores at 2.2 GHz
real 2m42.395s
user 10m56.020s
sys 0m33.980s
SBCL without hack, 8 cores at 2.2 GHz
real 2m57.945s
user 11m15.622s
sys 0m33.902s
SBCL with hack, 16 threads and cpu boost
real 1m17.841s
user 8m11.300s
sys 0m32.597s
SBCL without hack, 16 threads and cpu boost
real 1m25.606s
user 8m19.650s
sys 0m29.466s
===============
diff --git a/src/algebra/Makefile.in b/src/algebra/Makefile.in
index 9431e0c7..87451c42 100644
--- a/src/algebra/Makefile.in
+++ b/src/algebra/Makefile.in
@@ -462,11 +462,12 @@ stamp-oboo3: stamp-db
@if [ ! -f stamp-oboo3 ] ; then \
echo "Bootstrap object copy" ; \
for A in ${CATLIST} ${DOMLIST} ; do \
- cp $$A.NRLIB/$$A.$(FASLEXT) ${OUT}/$$A.$(FASLEXT) || exit
1 ; \
+ cp $$A.NRLIB/$$A.lsp ${OUT}/$$A.lsp || exit 1 ; \
done; \
for A in ${CATDOMS} ; do \
- cp $${A}-.NRLIB/$${A}-.$(FASLEXT)
${OUT}/$${A}-.$(FASLEXT) || exit 1 ; \
+ cp $${A}-.NRLIB/$${A}-.lsp ${OUT}/$${A}-.lsp || exit 1 ; \
done; \
+ cp *.NRLIB/*.$(FASLEXT) ${OUT}/ ; \
rm -rf *.NRLIB \
echo "Stage 3 object bootstrap (normal mode)" ; \
echo > oboo3.input ; \
@@ -477,8 +478,9 @@ stamp-oboo3: stamp-db
DAASE=./r7 ${INTERPSYS} ) || exit 1 ; \
echo "Stage 3 object copy" ; \
for A in ${DOMLIST} ; do \
- cp $$A.NRLIB/$$A.$(FASLEXT) ${OUT}/$$A.$(FASLEXT) || exit
1 ; \
+ cp $$A.NRLIB/$$A.lsp ${OUT}/$$A.lsp || exit 1 ; \
done ; \
+ cp *.NRLIB/*.$(FASLEXT) ${OUT}/ ; \
touch stamp-oboo3 ; \
fi
diff --git a/src/algebra/boo_db.input b/src/algebra/boo_db.input
index f65c96fd..cd0e4c9a 100644
--- a/src/algebra/boo_db.input
+++ b/src/algebra/boo_db.input
@@ -1,3 +1,4 @@
+)boot $skipCompileLisp := true
)boot $SaveParseOnly := true
)read komp_all.input
)boot processGlobals()
diff --git a/src/interp/nlib.lisp b/src/interp/nlib.lisp
index fa10d184..2b966f7d 100644
--- a/src/interp/nlib.lisp
+++ b/src/interp/nlib.lisp
@@ -220,7 +220,8 @@
(defun |compile_lib_file|(fn)
(if FRICAS-LISP::algebra-optimization
(proclaim (cons 'optimize FRICAS-LISP::algebra-optimization)))
- (compile-file fn))
+ (if (not (boundp '|$skipCompileLisp|)) (compile-file fn))
+)
;; (RDROPITEMS filearg keys) don't delete, used in files.spad
diff --git a/src/lisp/fricas-lisp.lisp b/src/lisp/fricas-lisp.lisp
index aeaa3537..ffa56d25 100644
--- a/src/lisp/fricas-lisp.lisp
+++ b/src/lisp/fricas-lisp.lisp
@@ -355,7 +355,10 @@ with this hack and will try to convince the GCL
crowd to fix this.
;;; (format *error-output* "entred load_quietly ~&")
#-:GCL
(handler-bind ((warning #'muffle-warning))
- (load f))
+ (load
+ (cond ((probe-file f) f)
+ ((probe-file (concatenate 'string f ".fasl")) f)
+ (t (compile-file (concatenate 'string
(string-right-trim ".fasl" f) ".lsp"))))))
#+:GCL
(load f)
;;; (format *error-output* "finished load_quietly ~&")
===============
On 7/25/22 23:06, Waldek Hebisch wrote:
On Mon, Jul 25, 2022 at 06:58:54AM +0800, Qian Yun wrote:
On 7/25/22 02:47, Waldek Hebisch wrote:
On Sun, Jul 24, 2022 at 09:29:24PM +0800, Qian Yun wrote:
As you know, the majority time of building FriCAS is to compile
over 1000 SPAD files to LSP files and then to LISP_BIN file.
Luckily, this step can be paralleled.
So on a multi-core machine, to reduce build time, the focus should
be on the serial part of build process.
The building of algebra/*daase is the most important serial part.
I'm not entirely familiar with the process, but it seems there
are a few stages to generate daase, and each stage requires to
compile many (if not full) spad files, and the compiled result
is discarded.
More precisely, the result is replaced by result of subsequent
compilation. But we compile them because we need to load
some of them (at it is tricky to decide if we can skip some).
Yes, the step "make stamp-oboo3" loads some files, but we can
use "compile-file before load" to just compile the needed files.
So I think in this step, it is wasteful to compile
these *lsp file to *fasl file, especially for SBCL with :SB-FASTEVAL.
Quite likely.
Another motivation for this compile speed optimization is that,
when I was compiling use ECL, the total time is over 20 minutes,
and I guess the daase step takes around 10 minutes.
Current trunk, ECL 16.1.2 with -j 10 (on quad core machine with
hyperthreading) needs:
real 16m42.871s
user 61m26.912s
sys 2m54.504s
Of that 350 sec goes into boo_db stage.
The following is the diff I used during experiment.
(This is without the compile-before-load optimization.)
With that boo_db stage goes down to 118 s. However,
build fails at end of spad compile stage (before
generating HyperDoc pages) and the time is:
real 13m42.144s
user 74m20.816s
sys 2m32.908s
so real time may be lower, but there is increase in
CPU time. I do not know why, maybe loading compiled
files is cheaper than loading Lisp...
--
You received this message because you are subscribed to the Google Groups "FriCAS -
computer algebra system" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/fricas-devel/1b433891-28f6-0c74-c915-7fd17fd1fcfe%40gmail.com.
