I recommend a lock-serving place in Pollen to manage concurrency for parallel rendering. Don't let multiple places try to read and/or write to the same file, and don't try to use filesystem locks.
You can use filesystem locks to manage concurrency, but it's not convenient. Part of the problem is that there's no blocking operation to take a lock (due to the underlying OS abstractions). You can also use atomic filesystem operations to achieve the effect of locks or transactions, but it's really difficult to get that right. For the compilation part of `raco setup`, parallel builds rely on a combination of atomic filesystem actions plus the main place acting as a lock server. That is, a place can only compile a file if it can get permission for the file from the main place, and it reports back to the main place when the file has been compiled. If you run two `raco setup`s or `raco pkg install`s at the same time, things can go wrong, because the lock server within one multi-place process is important. For the document-building part of `raco setup`, you're right that it relies mostly on independent document rendering. Pollen rendering is probably more like the compilation part of `raco setup`. Document rendering in `raco setup` uses a shared cross-reference database, but concurrency at that level is managed by database transactions (as implemented somehow in SQLite). At Thu, 13 Jun 2019 09:28:33 -0700, Matthew Butterick wrote: > To Pollen, I've recently added a parallel-rendering facility with places, in > emulation of how Racket docs are rendered in parallel. > > But it doesn't quite work — I get intermittent / inconsistent errors from the > filesystem (for instance, certain files that should exist do not, or certain > files that shouldn't exist, do) > > My intuition about places is weak. But it seems to me that Scribble renders > are largely independent (= happening in separate folders on the filesystem). > Whereas within a Pollen project, the rendered sources are often relying on > shared files (for instance, a "template.html"). So there is more direct > competition for use of certain files, which perhaps is Bad News. > > Is there a rule of thumb for how to manage these conflicts? For instance, is > it better to just avoid having places try to read or write from the same > files? Is there some filesystem-blocking technique that can be used to make > sure reads & writes don't collide? > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/racket-users/2A49317F-F4C8-4EF2-B144-E8846571 > DD7F%40mbtype.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/20190613170407.51E56650111%40mail-svr1.cs.utah.edu. For more options, visit https://groups.google.com/d/optout.
