l...@gnu.org (Ludovic Courtès) writes:
> taylanbayi...@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis:
>
>> + (with-target target
>> + (lambda ()
>> + (let ((mutex (make-mutex)))
>> + (par-for-each
>> + (lambda (file)
>> + (let ((go (scm->go file)))
>> + (unless (and (file-exists? go)
>> + (file-mtime<? file go))
>> + (with-mutex mutex
>> + (format #t " GUILEC ~s~%" file)
>> + (force-output))
>> + (compile-file file #:output-file go #:opts compile-options)
>> + (with-mutex mutex
>> + (format #t " WROTE ~s~%" go)
>> + (force-output)))))
>> + files)))))
I'm sorry I haven't been following this discussion closely, but let me
first say that the performance gains you've been able to achieve are
very exciting. Thanks for working on this!
Unfortunately, I have concerns:
A few people mentioned on IRC that when doing these concurrent
compilations within a single process, they sometimes see warnings like
this:
;;; WARNING (module #<module () 262a990> not in submodules table)
I haven't yet investigated, but my strong suspicion is that this is due
to the fact that Guile's module system is not thread safe. More
specifically, when a new module is created, it mutates the global
directory of modules in a way that is not thread safe.
New modules are created by 'compile-file', both for the module being
compiled and for any imported modules that haven't been previously
loaded. Unfortunately, this means that this approach of compiling files
in multiple threads within a single guile process is not safe. There
are likely to be random crashes and corruptions.
I suggest that for now, the best way can do safely is to adopt an
approach of running multiple Guile processes, where each one compiles
multiple files within a single thread.
Regards,
Mark
