On Tuesday 13 November 2001 12:05 pm, Ken Fox wrote: > > Are _all_ handles (PMC's, Strings, or whatever) that serve as > > root-set elements dynamically allocated? > > Nope. Part of the root set includes auto C PMC handles used by > extensions. You also have to think about hardware registers too > if GC is running in another thread.
I'm not worried about separate threads. "Shared" handles are already allowed to be slow and thus few and far between. Locking also alleviates most of the hangups. Most of my thinking involves running much of the GC within a worker thread (spawned by allocs), so different interpreters are, for the most part, independant of each other, even in garbage collection. My ideal is to only profile at alloc time, then hand off a job to a gc-thread, but so far there are more problems than worthwhile (especially with coping GCs). The alternative is to enforce read/write barriers on code and do all aspects of GCing in another thread. This avoids much pause-time, but all but requires multi-threading (something I'd rather not require for one-liners). > > IMHO the easiest way to code this is to define an API that > extension writers use to declare every PMC handle. This code > looks ugly and encourages bugs though so a lot of people don't > like it. Many systems treat the C stack as conservative roots. > Those systems flush the hardware registers to the C stack > before running the collector. The benefit is that handles > don't need to be declared. The cost is the non-portable code > used to scan the C stack and flush the registers. The various texts I'm reading suggest that memory management should be a black-box module. It's easier to debug and use.. They suggest that too large a percentage time is devoted to memory management (leak-finding), so the less XS-writers have to worry about memory, the more acceptance parrot will have IMHO. (Even if some performance or intermitant pausing is necessary... Though hopefully we'll be able to provide a choice between the two at run-time). > Another simple solution is to suspend the collector while in > extension code (or untrusted extension code). This works for > lots of cases, but falls apart badly in callback/event-loop > extensions that never return. Unfortunately, this would require hooks into the mem-mngr module, which violates it's black-box nature. Not to mention the points you make. > One nice thing about writing Perl extensions is that the "SV *" > can be stuck in odd places. For example, I put them in X Toolkit > and Motif widget "user data" slots. Once the "SV *" are stuck > in the slots I have no idea what happens to them. This works fine > because the ref count will keep the SV alive until the widget > dies. A hybrid ref count GC with pinned PMCs will work exactly > the same. If we get fancier though and either use movable > PMCs or tracing collectors, then my "user data" slots become > roots. That will crash and burn because I don't know how to > register them. I'd have to use a separate array for holding the > PMCs and then stick the array indexes into the widgets. So long as SV* equivalents are registered as foreign access, then XS modules shouldn't have to worry. Unfortunately this might cause memory leaks. Perhaps foreign access can utilize a reference count.. XS modules that don't directly utilize GC-able objects, register a newly allocated handle with foreign access, and are required to unregister it when they're done.. When a handle is unregistered as many times as it's registered, it's removed from the foreign access area, and thus it's contents are reclaimable. Only remaining problem is in how the handle was acquired (hense my original question), since the handle needs to be reclaimed somehow, since the following is possible: PMC* ptr = malloc(sizeof(PMC)); ptr = malloc(sizeof(PMC)); // memory leak If allocations instead come from gc_malloc - or more likely from the arena newPMC(), then once the PMC is unregistered from foreign-access, XS-code can safely ignore it. The only gotcha here is that XS code has to treat unregistering as if the PMC were being free'd (and optimizations will probably do exactly that). Heck, I'm even thinking that there should be two forms of newPMC, one to be used within the root-set, and one used externally. newPMC, and newPMCR (for registered). It's still too early to see what's a good direction. > > It's probably not worth making my job as an extension > writer easier if it slows/complicates the rest of the GC. Most > extension writers are pretty good at this sort of thing and > a special utility library for pseudo-pinned PMCs wouldn't be > any big deal. I don't really see a need for handles to be movable (especially if they're arena based), but reclaiming handles might require for some sort of extra record-fields. Since handles are fixed sizes and decently big (several integers worth), I'm wanting to prepend next/prev pointers to the structure and maintain from/to/free lists. A gc-invocation navigates the root set and moves those elements from the "from-list" to the "to-list". Remaining "from-list" items get appeneded to the "free-list", which for arena's is a constant time. Depending on the buffer-object reclaiming method, either the newly free'd handles will have their associated objects reclaimed, or the migrated "from->to" objects will have their associated objects copied. This is an example of something that can be done in a separate thread. Navigating the handles is only a constant overhead over navigating the root-set (which is almost always unavoidable). If the root-set for each thread isn't that large, then pauses won't be that great. The only problem with this approach is that I'd like to hide the "prev/next" pointers since they are part of the black-box, and not part of the utility of the handles. But this requires some uniformity on how handles are created (ergo always from the arena). Again, this is all in the early stages of research. -Michael
