Interesting conversation.. I want to add that #lang web-server does have file-boxes ( https://docs.racket-lang.org/web-server/stateless.html?q=web-server#%28part._lang%2Ffile-box%29) for when one might want to preserve data like you would in the store. To run across multiple machines I would imagine one could also write a file-box-ish approach to a database, but I haven't gotten there yet. I think one of the exciting aspects for me regarding stateless continuations is the idea of running on multiple servers with the only shared state being cookies and file-boxes. I imagine being able to easily scale with something like Amazon Lambda. Although I haven't tried that yet either, so maybe there are roadblocks there.
On Sun, Oct 14, 2018 at 7:55 PM George Neuner <gneun...@comcast.net> wrote: > Hi Philip, > > Sorry for the delay in responding ... busy day. > > > On 10/14/2018 7:09 AM, Philip McGrath wrote: > > On Sun, Oct 14, 2018 at 5:30 AM George Neuner <gneun...@comcast.net> > wrote: > >> You are absolutely correct that in typical usage globals are not closed >> over ... normally they are considered mutable shared "communication" >> state. >> >> However, I was speaking more generally: to be persisted long duration, >> and be made portable across separate runtime instances, the continuation >> must close over AND deep copy ANY VALUE the current call chain references. >> You don't (necessarily) want to restart the continuation a week later, on a >> different machine, with different values than when it started. >> > > I am definitely not an expert on continuations or PL generally, so I could > be very wrong. My understanding, though, has been that continuations not > closing over non-local variables is a property of continuations in general: > not only in a web context, and not only of the serializable continuations > from `#lang web-server`. (I used those in my example just because it is > easy to see what values they do in fact close over.) > > > In the most basic form, the continuation is just "what to do next" - e.g., > a pointer to the next code to be executed. That is [more or less] all that > is required for a local (within a function) escape continuation because > [barring stack corruption] all of the EC's stack context is guaranteed to > exist if/when the continuation is invoked, and that state is needed anyway > because the program will continue on regardless of how it leaves the scope > of the EC. > > For a non-local EC, you need to return to the stack frame that was current > when the continuation was created [the rest of the stack can be > discarded]. In any case, an EC can very compact - just a couple of > pointers. > > [Aside: a Lisp-style exception requires that the entire call stack be kept > until you know where execution will continue. You need to return to the > EC's call frame (to find the exception handler), but Lisp allows exceptions > to introspect/modify the environment and then to return control to the > point of the exception (where the intrinsic EC was invoked). Scheme > doesn't have this capability.] > > > General continuations - such as can implement coroutines - need to capture > much more state. The entire stack state of the current function call chain > at the point where the continuation is created must be preserved in order > to reenter the continuation. > [this is where stateful and stateless part company - more below] > > > Winding continuations are an extension of general continuations to the > requirements of dynamic-wind. In addition to whatever is required for a > general continuation, a winding continuation has preamble code that must be > executed whenever a continuation enters the winding scope, and postamble > code to be executed whenever (and how ever) execution leaves the winding > scope. > > > The long winded point here is that continuations have no canon > implementation - if you want more functionality from them, you need to > capture/preserve more state. > > > > The web-server is multi-threaded, and the stack state of each thread is > unique - so for each thread in which a continuation is created in the > stateful server, the thread's call stack must be preserved. > > The stateless server is a different case. CPS code doesn't use a call > stack ... or rather its "stack" only ever contains one useful frame - the > current one. So there's only one call frame per thread that needs to be > preserved rather than a (maybe large) stack of them. This is why stateless > server continuations - in general - use much less memory than stateful > server continuations. > > [The above is a bit simplistic because Scheme supports nested functions > with non-local variable access. [Scheme also supports 1st class closures > based on nested functions, but that is not what this is about.] To > support non-local access without a stack, CPS code needs to construct > shared environments on the fly as the program executes to communicate > values from producers to consumers, and these environments (where > applicable) also must be part of saved continuation state. Environment > construction is similar to, but not the same as closure conversion: as > with closures the code must be compiled to expect the variable(s) in the > external environment block rather than in the call frame - but unlike > closures, a new environment must be constructed each time a given producer > needs to communicate with consumers. > > The alternative to these "environments" is to pass everything by argument > - but doing that in CPS quickly leads to functions with huge argument > lists, and to functions having many "pass-thru" arguments that they don't > use. > > A related procedure done with stack-based implementations is sometimes > referred to as "stack flattening". When a function high in the call chain > must communicate with functions lower down in the chain, a shared > environment is allocated high on the stack and a pointer to it is passed as > an argument down through the chain allowing any lower function to access > it. The stack case is somewhat easier to analyze than the CPS case because > typically there are just a small number of functions involved. The CPS > case is more complicated: although any particular environment will have a > limited lifetime, often multiple different (logical) environments to > coexist simultaneously in a call chain of (potentially) unlimited length in > which the number of functions involved can be up to every function in the > program. It makes for an interesting compilation and runtime structuring > problem.] > > > The documentation for `#lang web-server` does emphatically warn that > >> the store is *not* serialized. If you rely on the store you will be >> taking huge risks. You will be assuming that the serialized continuation is >> invoked on the same server before the server is restarted or the memory is >> garbage collected. >> > > It is reasonable to assume a continuation will be invoked on the same > program instance. But assuming referenced (heap) objects will not be > garbage collected in the meantime IMO is not reasonable. That alone is a > reason NOT to use serialized continuations - better to use in-memory > continuations that preserve their complete (program-wide) state. > > > As far as I can see, serialized continuations from `#lang web-server`, for > better or for worse, have the same issues in these respects as the kinds of > manually RESTful web applications you might write in various languages with > names beginning in P, where you would marshal an ad-hoc representation of > the continuation into a query parameter, path segment, or hidden form > field. In that case, also, multiple server instances with different > server-side state could create chaos. If you are relying on that kind of > server-side state, though, your web application isn't really RESTful at > all. The web-server language doesn't prevent you from creating such > problems, but it shows you a principled way to avoid them: write functional > programs. > > > REST actually only constrains the client/server communication protocol to > being stateless ... it doesn't constrain the service itself to being > stateless. > > The problem with serialized continuations is that they don't save enough > state to restart the computation if the required state has been purged > (GC'd), or was not present to begin with in a different (identical code) > server instance. > > George > > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
