Re: GC design
On Sun, May 26, 2002 at 08:20:23AM -0700, Sean O'Rourke wrote: > I'm sure it's been discussed before somewhere, but why can't we guarantee > that we only do GC runs between ops? Each op would just have to guarantee > that all of its persistent data is reachable before it returns. It seems > like this would avoid the need for a neonate flag entirely. If you know how much memory the op is going to need, then you can run the GC just before executing the op to make sure that at least this amount of memory is available (and non fragmented). But this seems to be a pretty strong requirement. Otherwise, you should be able to run the GC whenever you allocate some memory. > > "Mike Lambert" wrote: > > > Another proposal is to walk the C stack. What about managing explicitly a stack of Parrot values? Something like this: /* Save the top of the stack */ struct parrot_frame *parent = parrot_stack; /* Create a variable "x" which contains Parrot values */ parrot_value x = 0; /* This is the variable */ struct parrot_frame frame;/* Allocate a stack frame */ frame.val = &x; /* Push the value into the stack */ frame.next = parrot_stack; parrot_stack = &frame; /* ... Body of the function */ parrot_stack = parent;/* Pop the stack */ return; /* Exit the function */ With a suitable set of macros, this should be manageable. -- Jerome
Re: Hashtable+GC problems
On Sun, May 19, 2002 at 10:43:20PM -0400, Mike Lambert wrote: [...] > But if you'll notice down below in the body, it uses > string_compare. Due to transcoding issues, that can trigger a full GC, > which means you need to restore_invariants inside that loop. And that > invalidates your 'chain' which you've already placed on the local C stack, > and restore_invariants can't fix that. In my opinion, there should be a low level string_compare which does not allocate anything. Instead, it should fail if its arguments do not have the same encoding. Of course, the string_compare opcode can be smarter and perform some transcoding. Then, the hashtable code should explicitely transcode the key before using it. -- Jerome
[netlabs #628] [PATCH] Make hash.c depend on parrot headers
# New Ticket Created by "Peter Gibbs" # Please include the string: [netlabs #628] # in the subject line of all future correspondence about this issue. # http://bugs6.perl.org/rt2/Ticket/Display.html?id=628 > Following patch adds dependencies entry for hash.c to Makefile. Stops things from getting very confusing when changing layout of String and/or Buffer structs! -- Peter Gibbs EmKel Systems Index: config/gen/makefiles/root.in === RCS file: /home/perlcvs/parrot/config/gen/makefiles/root.in,v retrieving revision 1.2 diff -u -r1.2 root.in --- config/gen/makefiles/root.in24 May 2002 16:34:19 - 1.2 +++ config/gen/makefiles/root.in27 May 2002 10:23:12 - @@ -292,6 +292,8 @@ pmc$(O) : $(GENERAL_H_FILES) +hash$(O) : $(GENERAL_H_FILES) + jit$(O) : $(GENERAL_H_FILES) jit_cpu$(O): $(GENERAL_H_FILES) $(INC)/jit_emit.h
Re: GC design
On Mon, 27 May 2002, Jerome Vouillon wrote: > On Sun, May 26, 2002 at 08:20:23AM -0700, Sean O'Rourke wrote: > > I'm sure it's been discussed before somewhere, but why can't we guarantee > > that we only do GC runs between ops? Each op would just have to guarantee > > that all of its persistent data is reachable before it returns. It seems > > like this would avoid the need for a neonate flag entirely. > > If you know how much memory the op is going to need, then you can run > the GC just before executing the op to make sure that at least this > amount of memory is available (and non fragmented). But this seems to > be a pretty strong requirement. But there are two kinds of "available" here: available without asking the operating system for more; and available period. If we're in the first situation, it seems reasonable to just ask the OS for a new block and keep going, noting that collecting soon would be a Good Thing. Our memory requirements have probably increased, so we'd be asking for more soon anyways, and if they haven't, we'll reclaim a lot of memory on the next collection, and only be "wasting" 8 or 16k. We can even try to give some back, though the OS probably won't take it. If we're in the second situation, then either (1) we've gone far too long without collectiong, or (2) we're in serious trouble anyways. And even in this situation, we can wriggle out of it most of the time by keeping some amount of "emergency" memory stashed away. This may not work on embedded systems, but not having developed for one, I hate to just throw away the idea based on my ill-founded suspicions. > Otherwise, you should be able to run the GC whenever you allocate some > memory. Since our memory-allocating routines return NULL quite a ways back up the call chain (all the way through e.g. string_grow()), here's another way to do this -- if allocation returns NULL all the way to an op function, it can make the things it wants to keep reachable from somewhere, do a collection, and retry. By the way, neither of string_grow()'s callers checks its return value now, which indicates to me that this may be error-prone. /s
[netlabs #629] [PATCH] Memory manager/garbage collector - major revision
# New Ticket Created by "Peter Gibbs" # Please include the string: [netlabs #629] # in the subject line of all future correspondence about this issue. # http://bugs6.perl.org/rt2/Ticket/Display.html?id=629 > Attached patch does some fairly radical things to the memory manager. 1) Cycle counter added to interpreter structure; incremented by opcodes that have something vaguely resembling a function call in their body. 2) Buffer-like objects have their date of birth (cycle counter value) recorded 3) Neonate flag is deprecated; functionality replaced by the above i.e. no buffer header will be freed during the same opcode as it was created 4) Free pool array replaced by a linked list within the arenas 5) Minimum/maximum ratio of memory pool physical size to used size controlled by new tuning parameters (fixed #define's for now) 6) DOD run performed during memory allocation if compaction will not reclaim enough memory The 'function-call like' logic (which is very primitive at present) for adding the cycle count increment into an op's body means that most simple opcodes should not incur additional overhead. Once an op starts calling functions, the additional overhead of a single increment is likely to be minor. After much thought and playing around, this seems to be the best answer to the problem of infant mortality. Dan's suggestion of reversing the logic of DOD runs would work for the pure infant mortality situation (except perhaps for the odd pathological op), but it still leaves problems with the dependency of buffer memory collection on prior dead object detection. These changes do cause a slight performance degradation, but I believe it is worth it for the overall simplification of transparent protection of the newborn. Performance can only be a secondary goal, after correct behaviour. The runaway memory usage of programs like zip and quicksort has hopefully been resolved; I have successfully run zip.pasm against core_ops.c. Note that the neonate flag has not been removed; if this patch is applied, all use of the neonate flag can be removed, followed by the flag itself. The flag is not used in the new GC logic. The problem of having buffer memory move around still remains; that should be addressed shortly. -- Peter Gibbs EmKel Systems
Re: [netlabs #629] [PATCH] Memory manager/garbage collector -major revision
At 4:33 PM + 5/27/02, "Peter Gibbs" (via RT) wrote: >Dan's suggestion of reversing the logic of >DOD runs would work for the pure infant mortality situation (except >perhaps for the odd pathological op), but it still leaves problems >with the dependency of buffer memory collection on prior dead object >detection. That dependency, as such, is in there on purpose, and it's not really a problem. It's a win with mutable strings, something we don't take as much advantage of as we should. >These changes do cause a slight performance degradation, but I believe it is >worth it for the overall simplification of transparent protection of the >newborn. Unfortunately I don't. I'm willing to live with a performance degredation in those spots where it's truly necessary, but only there. The performance hit has to be localized to those places where infant mortality is actually a problem. Those spots are currently reasonably rare, and I'm not sure that the hoops we're building to jump through (and I'm as guilty as anyone) are worth it. >Performance can only be a secondary goal, after correct behaviour. But performance has to be given up as grudgingly as possible. -- Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
[COMMIT] Added preprocessor layer to newasm.
newasm now handles constants, macros, and local labels within. Here's a short test program that exercises all three featurs. Doesn't do much, but here it is: print "Everything else is as usual.\n" ..macro MyMacro ( FOO, Bar ) # Macros are now prepended with '.'. print .FOO # . also prepends our arguments add I3,I3,.BAR ..local $MyLabel: print "Label\n" # And label definitions if I3,.$MyLabel branch GlobalLabel ..endm print "Some stuff\n" .constant HelloWorld "Hello, World!\n" set S0,.HelloWorld GlobalLabel: print "Infinite loop here.\n" print "About to call the macro.\n" .MyMacro(S0,I3) # Invoke the macro. Note that the macro branches # to -before- this macro, so it's an infinite loop. print "We'll never get here, but here's the value of PerlArray: " print .PerlArray # It's a constant, but predefined. new P0,.PerlArray set P0[I1],.HelloWorld # Of course, keyed access still works. end Documentation in newasm has been updated, and the documentation should help explain things. Macros are not allowed to be recursive, and must be defined before they can be used. Same thing with constants. Also, note that -any- text of the form '.Foo' is a fair candidate for expansion. Also, they don't expand recursively. '.constant FOO "blah"\n.constant BAR "Hey, .FOO"' won't do what you want, sadly. It's passed the tests that I managed to change over to the new style. Feel free to report bugs and make your own changes. The most clamor that I've heard after this seems to be an all-Perl bytecode generator, so that's what I'll work on next. -- Jeff <[EMAIL PROTECTED]>
