On Mon, 29 Aug 2022 at 21:52, Scott Kostyshak <skost...@lyx.org> wrote:
> I used the hacks just to get an idea of whether such parallelization would > actually lead to a speed-up. It does. For example, on an 8-core machine, > the document files (user guide, etc.) export to .tex about 4x faster. This > isn't that helpful from a user perspective, because the main bottle-neck is > compiling to PDF, not the LaTeX generation. But it's still cool to see that > it actually does have a real effect. It may very well be helpful for users, I'm especially thinking about the DocBook export: I find it quite slow compared to LaTeX code generation (ballpark estimate: twice to thrice as slow). What's more important for users is that LyX doesn't do anything further from the DocBook file (except when generating ePub), which makes it more noticeable. Any improvement there would be great! But it points to another area where refactoring could be important before merging your change: having less redundancy between the generators. I believe a large part of the performance discrepancy should be solveable by more careful optimisation of the code (the current focus has been on correctness). I think that most of it came when I started generating some parts twice to get the correct output (including generating LaTeX and parsing roughly parts of it). > If I protect it with a mutex, then things work much better. But that's > exactly the code that I need to run in parallel for my knitr child > documents to export in parallel. Could someone explain intuitively why this > code is not thread-safe? Is there any hope of making it thread-safe without > major surgery? > Another point of view: wouldn't this major surgery bring real improvements to the code base? For now, I found that the way the generators are written is ad-hoc and has evolved over the years, with the state stored in OutputParams getting larger and larger (and I got lost more than a few times in its updates). What you could ship quite quickly, in my opinion, is parallelising the operations on child documents. I think that part should almost be thread-safe. It would solve your problem too, while we could get some feedback from users (concurrency is always a tricky topic...).
-- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
