On Sun, 5 Jan 2020 19:22:14 +0100 Zelphir Kaltstahl <zelphirkaltst...@posteo.de> wrote: > I think the decision tree calculations, which I want to parallelize, are > not I/O related. However, I am not quite sure I understand the whole > suspendable port thing, but here is what I think it is: > > When some I/O happens in a fiber, the fiber is still able to pause > (suspend, yield, …) at that point, because the suspendable ports work in > such a way, that no harm is done in doing so. Later the I/O work can be > resumed (woken up from suspension). This quality had to be built into > Guile first, before fibers were able to take advantage of it. > > Is this correct?
Yes, suspendable ports were first implemented in guile-2.2. They are used by 8-sync (https://www.gnu.org/software/8sync/), guile-a-sync2 (https://github.com/ChrisVine/guile-a-sync2/wiki) and fibers, and possibly by other libraries. The basic arrangement is that if a port's file descriptor is not ready, then its continuation is saved, and is resumed by the scheduler when it becomes ready. fibers use epoll rather than POSIX select or poll for this. > But I do not understand, why this is not the case with normal OS > threads. Maybe it can be done but is not convenient or difficult to get > right, to work with suspendable ports, when not using the fibers library? > > And why is simple-format not "suspendable-port-safe"? (What does a > procedure need to do, in order to be "suspendable-port-safe"?) simple-format does not suspend as it is implemented in C in libguile and its continuation is not available to scheme code. There is a list of those of guile's i/o procedures which do (and which do not) suspend here, in the second and third paragraphs, although it does not mention format/simple-format: https://github.com/ChrisVine/guile-a-sync2/wiki/await-ports (That library has nothing to do with fibers, but as mentioned above it happens to use suspendable ports for similar purposes): None of this is likely to be relevant to your use case. [snip] > With the parallel forms, isn't it the case, that at each call of such a > form, new OS threads are created? In this case it might be a good idea > to create a fibers scheduler and reuse it, if that is possible, so that > I do not need to create my own process pool kind of thingy. guile's parallel forms are implemented using futures, which use an internal thread pool according to this: https://www.gnu.org/software/guile/docs/master/guile.html/Futures.html#Futures "Internally, a fixed-size pool of threads is used to evaluate futures, such that offloading the evaluation of an expression to another thread doesn’t incur thread creation costs. By default, the pool contains one thread per available CPU core, minus one, to account for the main thread." Chris
