This week's Perl Summary

[Piers Cawley]

The Perl 6 Summary for the week ending 20021229
This is not your normal summary. It's been Christmas, things have been
quiet, I've been concentrating on mince pies, roast goose and all that
other good stuff. Normal service will be resumed next week.

Acknowledgements
*   Larry Wall is just wonderful. Thanks for you ongoing design work and
for your injections of clarity into the mailing lists. Not that I
want to nag or anything, but could we have a few more apocalypses
this year?

*   Damian Conway is madder than a treeful of very mad fish, but in a
good way. Thanks for your patience and continuing effort -- it's
always a pleasure to read your posts, especially the ones where you
implement Hard Stuff in about 10 lines of lucid Perl 6.

*   Dan Sugalski is a virtual machine designer of taste and
descrimination. Thanks for Parrot, it's grrreat.

*   Leopold Tötsch is the Patch Monster! Thanks for your staggering
Parrot patch output.

*   Leon Brocard is not just a running joke. But thanks for making
it easy to keep the joke running.

*   Everyone who answered the questionnaire at any time, thanks.

*   Thanks to everyone who has given me feedback as a result of these
summaries. It's really good to know that people finding these things
useful.

*   Everyone who has donated to the Perl Foundation (at
), whether in response to these
summaries or not.

*   Gill Cawley, for obvious reasons.

I've missed out a pile of people I know, but you're either partial or
you end up doing a massively long, bad Oscar acceptance speech kind of
list.

Have a great 2003 folks.

Re: Infant mortality

[Leopold Toetsch]

Steve Fink wrote:


On Dec-31, Leopold Toetsch wrote:




I think, it moves the problems just around with a lot of overhead. E.g. 
cloning a PerlArray of 10^6 entries would need 1000 generations 


I don't understand. The outer clone belongs to generation N. Then if
you called clone() individually on each PMC within the array, then by
default you wouldn't create any more generations at all. 


Ah, ok. I thougt that at each dod run a new generation is started. But 
you seem to manage generation count manually.


I'm not convinced that's ever possible. What if an object overrides
its clone()? 


The vtable->clone in core.ops would be surrounded by my proposed 
do_dod_run/block_DOD, so the clone function could be anything.


Wait. Dumb question: is clone a shallow or deep copy?



Deep and recursive. BTW recursively marking was declined becaus of stack 
usage - what about clone?


By rearraning code and disabling DOD during clone (when needed), it 
should be possible to avoid walking the processor stack/regs alltogether.


Yes, I'm hoping there's some solution where things that have full
knowledge of what might happen can make use of it and gain speed in
the common case, while there's still a safe mechanism for things that
are too difficult or impossible to predict in advance. So we can start
with the easy approach, and then optimize them into the more detailed
approach if needed. Or something. My hands are getting tired from all
the waving around.


I think its time to do some coding.
leo

Re: Infant mortality

[Leopold Toetsch]

Steve Fink wrote:


To anyone out there who is thinking of a Grand Unified Infant
Mortality Solution, here's another thing that vastly complicates
things, and that we don't yet have a workable solution for yet: prompt
finalization.

  my %x;
  { # start scope
my $timer1 = Timer->new();
my $timer2 = Timer->new();
$x{timer} = $timer2;
...
  } # $timer1->DESTROY should be called
# $timer2->DESTROY should NOT be called

Right now, the only correct implementation of this that has been
proposed is to do a full DOD run at the end of every single scope. 


Active destroying (did I say: "We don't need it" ;-)

The question is: does the HL know, that $timer1 needs to be destroyed, 
while $timer2 is still refererenced outside the current scope?

... We
can do minor optimizations of that, by for example suppressing the DOD
if no object that implements a DESTROY method is live, but that's it.



Or a more specific destroy flag or a separate object list for objects, 
which are not allowed to live beyond the end of a scope?


At the moment, this threatens to make perl6 the *slowest* language
hosted by Parrot! (Since we haven't implemented any other languages
yet that need this.) 


Doing a full DOD run at the end of a scope is not an option.


... It is possible that this problem will be fixed by
weakening the finalization guarantees in the language definition, but
it's definitely something to worry about.



Weakening guarantees is probably not an option: file handles have the 
same problem - they should AFAIK be closed, when going out of scope.


The connection to the infant mortality problem is just that some
solutions might make this easier. Or harder. For example, the current
conservative stackwalk breaks even the painfully slow solution: there
might be a stale pointer somewhere on the C stack (or in a register,
or a saved register window) that keeps the DESTROYable object alive
unnaturally.


... which seems to indicate, that stack/register walking is unusable for 
such obbjects, which are not allowed to be live.

leo

Re: Infant mortality

[Leopold Toetsch]

Steve Fink wrote:


Another (maybe silly) possibility suggested itself to me based on a
private mail re: infant mortality from Thomas Whateley: could we try
optimistic allocations?




  if (alloc_object() == NULL) {
undo everything
do_DOD_run
interp->on_alloc_fail = CRASH_AND_BURN
start over
  }


What if e.g. clone needs more headers then one dod run can provide? With 
CRASH_AND_BURN it would not succeed. With RETURN_NULL it would start 
over and over again, until objects_per_alloc increases the needed header 
count.

leo

Re: Infant mortality

[Leopold Toetsch]

Matt Fowles wrote:


All~

Parrot_do_at_most_N_DOD/GC(1);
do_clone
Parrot_unblock_DOD/GC(-1);

This would allow it to do one DOD run if it realized that it needed it part
way through the clone, but would never do more.  


Very interesting approach. Yep. No need to check for probably needed 
resources. But the problem here is - this works only for the approach 
"setting the live (or neonate) FLAG for new objects - only one DOD run 
allowed".

The problem with this approach is, that you have far to many live 
objects laying around. DOD runs are rare.

I did test it with bench_newp.pasm: loops per second decreased from ~770 
to 7, PMC count was 100fold so DOD runs took a long time.

This would work only, in combination with resetting live_FLAG for 
already anchored objects, which share the problem with other neonate 
flag approaches: How and when to reset the flag.


Matt


leo

Re: Infant mortality

[Leopold Toetsch]

Steve Fink wrote:




On Jan-01, Leopold Toetsch wrote:



Yep. The goal can only be, to either have *all* code paths that might 
trigger DOD checked, or use one of the mentioned strategies. Though 
reducing possibilities of infant mortatility by early anchoring is IMHO 
always a good thing: it would e.g. help active pinning.


Could you expand on that? What's "active pinning"?



Proposed "Solution 3: Explicity root set augmentation"


leo

[perl #19668] [PATCH] infant mortality #1

[via RT]

# New Ticket Created by  Leopold Toetsch 
# Please include the string:  [perl #19668]
# in the subject line of all future correspondence about this issue. 
# http://rt.perl.org/rt2/Ticket/Display.html?id=19668 >


Attached patch changes a view places, which cause problems running w/o 
trace_system_areas().

With this all parrot tests pass here (i386/linux) with or without 
--gc-debug. Some perl6 tests related to try/catch are failing.

Please check this on different platforms, TIA.

leo



-- attachment  1 --
url: http://rt.perl.org/rt2/attach/46715/36710/7ee026/infant-mort-1.patch


--- parrot/classes/default.pmc  Mon Dec 30 18:38:57 2002
+++ parrot-leo/classes/default.pmc  Thu Jan  2 12:44:10 2003
@@ -65,7 +65,8 @@
  SELF->metadata, p_key, value, NULL);
} else {
   /* first make new hash */
- SELF->metadata = pmc_new(interpreter, enum_class_PerlHash);
+ SELF->metadata = pmc_new_noinit(interpreter, enum_class_PerlHash);
+ SELF->metadata->vtable->init(interpreter, SELF->metadata);
  /* then the key, else it vanishes with --gc-debug */
   p_key = key_new_string(interpreter, key);
  SELF->metadata->vtable->set_pmc_keyed(interpreter,
--- parrot/dod.cMon Dec 30 11:47:25 2002
+++ parrot-leo/dod.cThu Jan  2 12:18:40 2003
@@ -22,8 +22,6 @@
 #endif
 
 static size_t find_common_mask(size_t val1, size_t val2);
-static void trace_system_stack(struct Parrot_Interp *);
-
 
 void pobject_lives(struct Parrot_Interp *interpreter, PObj *obj)
 {
@@ -93,7 +91,7 @@
 #if ! DISABLE_GC_DEBUG
 CONSERVATIVE_POINTER_CHASING = 1;
 #endif
-trace_system_areas(interpreter);
+/* trace_system_areas(interpreter); */
 #if ! DISABLE_GC_DEBUG
 CONSERVATIVE_POINTER_CHASING = 0;
 #endif
--- parrot/hash.c   Fri Dec 27 10:34:28 2002
+++ parrot-leo/hash.c   Thu Jan  2 12:34:27 2003
@@ -418,6 +418,8 @@
 hash_clone(struct Parrot_Interp *interp, HASH *hash, HASH **dest)
 {
 HashIndex i;
+
+Parrot_block_DOD(interp);
 new_hash(interp, dest);
 for (i = 0; i <= hash->max_chain; i++) {
 BucketIndex bi = lookupBucketIndex(hash, i);
@@ -456,6 +458,7 @@
 bi = b->next;
 }
 }
+Parrot_unblock_DOD(interp);
 }
 
 /*
--- parrot/resources.c  Fri Dec 27 10:34:28 2002
+++ parrot-leo/resources.c  Tue Dec 31 14:26:26 2002
@@ -45,6 +45,8 @@
 new_block = mem_sys_allocate_zeroed(sizeof(struct Memory_Block) +
 alloc_size + 32);
 if (!new_block) {
+fprintf(stderr, "out of mem allocsize = %d\n", (int)alloc_size+32);
+exit(1);
 return NULL;
 }
 
@@ -110,25 +112,25 @@
 }
 }
 if (pool->top_block->free < size) {
+Parrot_do_dod_run(interpreter);
 /* Compact the pool if allowed and worthwhile */
 if (pool->compact) {
 /* don't bother reclaiming if its just chicken feed */
-if ((pool->possibly_reclaimable + pool->guaranteed_reclaimable) / 2
-> (size_t)(pool->total_allocated * pool->reclaim_factor)
+if (pool->possibly_reclaimable * pool->reclaim_factor
+> size
 /* don't bother reclaiming if it won't even be enough */
-&& (pool->guaranteed_reclaimable > size)
+|| (pool->guaranteed_reclaimable > size)
 ) {
 (*pool->compact) (interpreter, pool);
 }
-else {
-Parrot_do_dod_run(interpreter);
-}
 
 }
 if (pool->top_block->free < size) {
 alloc_new_block(interpreter, size, pool);
 interpreter->mem_allocs_since_last_collect++;
 if (pool->top_block->free < size) {
+fprintf(stderr, "out of mem\n");
+exit(1);
 return NULL;
 }
 }
--- parrot/spf_render.c Tue Dec 17 08:30:55 2002
+++ parrot-leo/spf_render.c Thu Jan  2 12:28:59 2003
@@ -228,6 +228,7 @@
 char tc[PARROT_SPRINTF_BUFFER_SIZE];
 
 
+Parrot_block_DOD(interpreter);
 for (i = old = len = 0; i < (INTVAL) string_length(pat); i++) {
 if (string_ord(pat, i) == '%') {/* % */
 if (len) {
@@ -663,6 +664,7 @@
 string_append(interpreter, targ, substr, 0);
 }
 
+Parrot_unblock_DOD(interpreter);
 return targ;
 }
 
--- parrot/string.c Thu Dec 26 12:32:35 2002
+++ parrot-leo/string.c Thu Jan  2 12:41:12 2003
@@ -244,7 +244,12 @@
 PObj_bufstart_external_SET(s);
 }
 else {
+/* allocate_string can trigger DOD, which destroys above allocated
+ * string header w/o stack_walk
+ */
+Parrot_block_DOD(interpreter);
 Parrot_allocate_string(interpreter, s, len);
+Parrot_unblock_DOD(interpreter);
 }
 s->encoding = encoding;
 s->type = type;

Re: Infant mortality

[Jason Gloudon]

On Wed, Jan 01, 2003 at 08:09:05PM -0800, Steve Fink wrote:

> To anyone out there who is thinking of a Grand Unified Infant
> Mortality Solution, here's another thing that vastly complicates
> things, and that we don't yet have a workable solution for yet: prompt
> finalization.

Timely finalization has no perfect solution even under the best conditions
(compiler support. etc.), this is why Java and .NET offer no promises about
timely finalization.

.
.

> The connection to the infant mortality problem is just that some
> solutions might make this easier. Or harder. For example, the current
> conservative stackwalk breaks even the painfully slow solution: there

As long as you're using a tracing collector, you cannot provide a general
guarantee of timely finalization without running a full sweep of the 'heap'.
Even if you do not have to walk the hardware stack that's too costly to be
practical.

The best partial solution to early finalization is compile-time tracing of
possible references by the compiler which can explicitly generate the
appropriate DESTROY calls.

-- 
Jason

[perl #19670] [PASM] bug in i/o (read on stdin)

[via RT]

# New Ticket Created by  Jerome Quelin 
# Please include the string:  [perl #19670]
# in the subject line of all future correspondence about this issue. 
# http://rt.perl.org/rt2/Ticket/Display.html?id=19670 >


Hi,

The following code does not work as it should:
read S0, 0, 1
print ">"
print S0
print "<\n"
end

$ perl ../../assemble.pl q.pasm > q && ../../parrot q
abcde
><
$ bcde
zsh: command not found: bcde

So it seems that Parrot "reads" the first char, as I want, but does not 
stuff it in S0. Or maybe I'm wrong, and should "open" STDIN before 
reading it via a file PMC?

Another thing. Could one turn off bufferization, so that "read S0, 0, 1" 
only reads one char without slurping the whole line till the carriage 
return?

Jerome
-- 
[EMAIL PROTECTED]

[perl #19671] [PATCH] befunge debugger supports the "delete" instruction

[via RT]

# New Ticket Created by  Jerome Quelin 
# Please include the string:  [perl #19671]
# in the subject line of all future correspondence about this issue. 
# http://rt.perl.org/rt2/Ticket/Display.html?id=19671 >


The previous befunge patch enabled befunge to set breakpoints, this one 
allows one to clear those breakpoints (thanks to Leo who fixed the 
PerlHash vtable).

Jerome
-- 
[EMAIL PROTECTED]


-- attachment  1 --
url: 
http://rt.perl.org/rt2/attach/46731/36719/ffd307/befunge_debugger_clear_breakpoints.patch


? befunge.pbc
Index: Changes
===
RCS file: /cvs/public/parrot/languages/befunge/Changes,v
retrieving revision 1.6
diff -u -d -r1.6 Changes
--- Changes	30 Dec 2002 20:55:44 -	1.6
+++ Changes	2 Jan 2003 16:53:13 -
@@ -1,5 +1,9 @@
 Revision history for a Befunge interpreter written for Parrot.
 
+0.1.2 Thu Jan  2 17:50:25 CET 2003
+- new debugger instruction: "delete", that allows to clear
+  breakpoints (those created by the "break" instruction)
+
 0.1.1 Mon Dec 30 18:12:34 CET 2002
 - debugger now accepts breakpoints: either on instructions
   (characters), or on a specified location (x,y), or on a
Index: README
===
RCS file: /cvs/public/parrot/languages/befunge/README,v
retrieving revision 1.8
diff -u -d -r1.8 README
--- README	30 Dec 2002 20:55:44 -	1.8
+++ README	2 Jan 2003 16:53:13 -
@@ -1,6 +1,6 @@
 DESCRIPTION
 ---
-This is a Befunge interpreter written in Parrot assembler, version 0.1.1
+This is a Befunge interpreter written in Parrot assembler, version 0.1.2
 
 This interpreter should be Befunge-93 compliant. This means the
 playfield is limited to 80x25. This should also mean that the torus
Index: debug.pasm
===
RCS file: /cvs/public/parrot/languages/befunge/debug.pasm,v
retrieving revision 1.4
diff -u -d -r1.4 debug.pasm
--- debug.pasm	30 Dec 2002 20:55:44 -	1.4
+++ debug.pasm	2 Jan 2003 16:53:13 -
@@ -84,6 +84,7 @@
 substr S11, S10, 0, 5
 eq S11, "break", DEBUG_INTERACT_BREAK
 substr S11, S10, 0, 6
+eq S11, "delete", DEBUG_INTERACT_DELETE
 eq S11, "status", DEBUG_INTERACT_STATUS
 substr S11, S10, 0, 7
 eq S11, "restart", DEBUG_INTERACT_RESTART
@@ -101,6 +102,13 @@
 DEBUG_INTERACT_CONTINUE:
 set P3[0], 0# do not stop at next instruction
 branch DEBUG_INTERACT_END
+DEBUG_INTERACT_DELETE:
+substr S11, S10, 0, 7, ""
+pushp
+set P4, P3[1]
+delete P4[S10]
+popp
+branch DEBUG_INTERACT
 DEBUG_INTERACT_DUMP:
 bsr DEBUG_DUMP_PLAYFIELD
 branch DEBUG_INTERACT
@@ -112,6 +120,10 @@
 print " break x,y - set a breakpoint at coords (x,y)\n"
 print " break c:x - set a breakpoint on column x\n"
 print " break r:y - set a breakpoint on row y\n"
+print " delete c  - delete breakpoint on character c\n"
+print " delete x,y- delete breakpoint at coords (x,y)\n"
+print " delete c:x- delete breakpoint on column x\n"
+print " delete r:y- delete breakpoint on row y\n"
 print " list  - list breakpoints\n"
 print " next  - step one befunge instruction\n"
 print " continue  - resume execution\n"

[PASM] PerlHash and keys

[Jerome Quelin]

Hi there,

How can one retrieve the keys of a PerlHash in Parrot assembly? Is there 
a way to traverse a hash?

Jerome
-- 
[EMAIL PROTECTED]

Re: Infant mortality

[Leopold Toetsch]

Jason Gloudon wrote:


On Wed, Jan 01, 2003 at 08:09:05PM -0800, Steve Fink wrote:



To anyone out there who is thinking of a Grand Unified Infant
Mortality Solution, here's another thing that vastly complicates
things, and that we don't yet have a workable solution for yet: prompt
finalization.



As long as you're using a tracing collector, you cannot provide a general
guarantee of timely finalization without running a full sweep of the 'heap'.
Even if you do not have to walk the hardware stack that's too costly to be
practical.



Yep. s. google: "active destruction group:perl.perl6.internals" Subject 
"DOD etc"


The best partial solution to early finalization is compile-time tracing of
possible references by the compiler which can explicitly generate the
appropriate DESTROY calls.


... if the compiler can track all usage, which I doubt.

leo

Re: Infant mortality

[Angel Faus]

>
> The best partial solution to early finalization is compile-time
> tracing of possible references by the compiler which can explicitly
> generate the appropriate DESTROY calls.
>

What about refcounting + real GC?

refcounting will free most of the objects as soon as possible, and the 
real GC system makes sure that even cyclical references are 
eventually freed..

-angel 
(who is not really paying atention at all)

Re: Infant mortality

[Dan Sugalski]

At 10:24 PM +0100 1/2/03, Angel Faus wrote:

 >

 The best partial solution to early finalization is compile-time
 tracing of possible references by the compiler which can explicitly
 generate the appropriate DESTROY calls.



What about refcounting + real GC?


Yech, refcounting is one of the things we're trying to avoid. The 
cost and error-prone-ness is its big downfall.

Anyway, to respond to the whole thread here, as I dig through my mail backlog:

1) We're not generally guaranteeing timely destruction
2) If something *wants* timely destruction, it can push a DOD run as 
a scope exit action. If something needing scope exit cleanup actually 
escapes its construction scope, odds are it'll live for some 
indeterminate time anyway, so that's fine
3) If #2 isn't sufficient, we can consider having a "needs timely 
cleanup" flag with corresponding interpreter counter, and on scope 
exit if the counter is non-zero a DOD run can be scheduled, with the 
interpreter handling the counter.

While I don't really like walking the system stack and probing the 
registers on DOD runs, the cost seems generally lower and less 
error-prone than explicit save/unsave actions that happen every time 
a PMC or buffer is allocated, and I'd like to start working in some 
sort of adaptive allocation/collection code so the DOD/GC doesn't 
fire off every time there's temporary resource starvation.
--
Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

RE: Infant mortality [x-adr][x-bayes]

[Garrett Goebel]

From: Angel Faus [mailto:[EMAIL PROTECTED]]
> 
> What about refcounting + real GC?
> 
> refcounting will free most of the objects as soon as 
> possible, and the real GC system makes sure that even
> cyclical references are eventually freed..

I believe this was covered in that same Re: DOC etc thread... 

http://groups.google.com/groups?q=refcount+GC&meta=group%3Dperl.perl6.intern
als

--
Garrett Goebel
IS Development Specialist

ScriptPro   Direct: 913.403.5261
5828 Reeds Road   Main: 913.384.1008
Mission, KS 66202  Fax: 913.384.2180
www.scriptpro.com  [EMAIL PROTECTED]

Re: [PASM] PerlHash and keys

[Leopold Toetsch]

Jerome Quelin wrote:


Hi there,

How can one retrieve the keys of a PerlHash in Parrot assembly? Is there 
a way to traverse a hash?


Not yet.
There is a nextkey_keyed mentioned in pdd02_vtables.pod, which would 
almost be all to implement aggregate iterators. Missing is IMHO how to 
reset (start) an iteration.

Also not too long ago, there was some proposal WRT an iterator class. 

This proposal needed more vtable functions.


Also not specified: how to attach an iterator to an aggregate.


Jerome


leo, who cc'ed Dan for clarification ;-)

Re: Infant mortality

[Leopold Toetsch]

Angel Faus wrote:



What about refcounting + real GC?



As soon as you start to refcount objects, which need active destroying, 
you end up refcounting all objects (a timer may be stored in an 
aggregate, passed to subs, may have references, ...).


So this is not an option, except changing everything to refcounting, 

which is a PITA.


-angel 

leo

PRE / POST in loops

[Arthur Bergman]

Hello,

I just got a question from Lee Pumphret regarding Hook::Scope POST in 
loops.

Currently it treats every iteration as a scope entry and scope exit so.

for(1..3) {
	POST { print 2 };
	print 1;
}

will print "121212",

Since perl6 seems to have a NEXT {} block for doing this, how is POST 
and CATCH supposed to
be used in conjunction with loop scopes.

Arthur

Status: Next Sections (Arrays! Hashes!)

[Michael Lazzaro]

OK, back from hiatus.  We paused a while over the holidays to allow A6 
to come to a simmer... it's probably still not going to come off the 
stove *too* soon, though, so I suggest we move on.

When we left off, we were talking about context vs. type on p6l.  That 
didn't really go anywhere, and won't until after A6, probably.  So 
let's skip it.

STATUS UPDATE:

(1) We have some good stuff on the simplest types of scalar values; 
namely, numerics and strings.

(2) We don't have docs on the other scalar types (Ref, Code, etc.)  
That's OK, we can fill them in later.

(3) I have some fluff on variables/types/values loosely adapted from
  http://cog.cognitivity.com/perl6/val.html
(It needs some proofing and cleanup, and I need to make a tree 
structure out of it, but I think we can make some of the initial dumb 
"what's the word 'variable' mean" subsections using that text.)

After I fixup (3), I'll have a crude document put together that's the 
tree-ified composite of everything we have so far, which I'll post.

CURRENT TASKS:

All of that is still only a small part of what is in, or was implied 
by, A2.  Assuming we can skip the rest of the scalar stuff, that leads 
us to arrays and hashes.  Here's a *very* crude outline of what needs 
to be covered for arrays:

- The Array Type
  [] declaring an Array
  [] general case: C
  [] simplified: C or C
  [] simplified: C
  [] complex types
  [] C, etc.
  [] Perl6 builtin Array types  (just the one?)
  [] subclassing Array
  [] to store a specific type
  [] ex: C
  [] to create an alternate implementation
  (overview; see elsewhere for details)

  [] arrays vs. references to arrays
  [] C vs. C
  [] an array returns a ref to itself in scalar context

  [] declaring literal arrays
  [] () vs. []
  [] [1,2,3] means scalar(list(1,2,3))
 (or is it scalar(array(1,2,3))?)
  [] .. (range operator)
  [] : (step operator)
  [] lazy
  [] ( 1 .. Inf )

  [] accessing
  [] by index []
  [] @arr[-1]   (negative indices)
  [] slices
  [] using indices
  [] using ranges
  [] using steps
  [] *@arr to flatten
  [] lazily
  [] ( 1 .. Inf )
  [] ($first,@rest) = ( 1 .. Inf )  (needs to work)

  [] Using an array in other contexts
  [] Using an array in scalar context
  [] boolean
  [] numeric
  [] string
  [] C<"blah @foo blah">
  [] rule
  [] Using an array in list/array context
  [] the difference between lists and arrays
  [] Using an array in hash context


Hashes should have a similar-but-larger outline.  Am I missing anything?

If anyone wants to take this on, or just parts of it, let us know.  If 
anyone has any big questions we need resolution for, plz post here or 
on perl6-language.  I know I'll have a few.

MikeL

Re: Infant mortality

[Nicholas Clark]

On Thu, Jan 02, 2003 at 08:12:45PM +0100, Leopold Toetsch wrote:
> Angel Faus wrote:
> 
> >
> >What about refcounting + real GC?
> 
> 
> As soon as you start to refcount objects, which need active destroying, 
> you end up refcounting all objects (a timer may be stored in an 
> aggregate, passed to subs, may have references, ...).

Well, I thought that you only needed to refcount anything (and everything)
that currently contains an object that needs active destroying (where
references, subs and anything else that holds a reference counts as
"containing", because I can't think of a better word)

Effectively everything currently holding an object needing active
destruction becomes an object needing active destruction, (for the duration,
but no longer), and so on all the way back up to the root set.

Whether in practical terms this turns out to mean virtually everything
anyway, I don't know.

> So this is not an option, except changing everything to refcounting, 
> 
> which is a PITA.

Quite.

We're not guaranteeing perfection, though, are we? Just better than perl5?

[ie things wanting timely destruction get their destructor called at the
correct time, unless you've been silly enough to send them off into a
refcount loop. In which case they will get spotted at some point probably
before program exit, compared with perl5 which only spots them if your
program exits]

Nicholas Clark

Re: Returning new PMCs (again)

[Dan Sugalski]

At 11:18 PM -0500 12/31/02, David Robins wrote:

On Tue, 31 Dec 2002, Dan Sugalski wrote:


 >I don't think any ops do that presently (that would take a PMC** param).
 Oh, sure, lots do. Remember the ops get a pointer to the PMC
 register, which is itself a pointer. Whatever you stuff in there is
 what the register is set to.


Right, I meant PMC vtable, not ops.


Ah, OK, that's different. :)

Ripping out the rest of the stuff, this boils down to a matter of 
behaviour. What you're looking for is "perform operation and return 
value" semantics, while the vtables specify "perform operation and 
assign value" semantics. This is generally the more-likely to be used 
way, and we've gone with it for performance reasons.

In those cases where there is no destination of the assignment, you 
need to create an empty PMC of the appropriate type, or something 
that can become the appropriate type, and pass it into the vtable 
method.

This also brings up some issues of object behaviour, which I think I 
need to specify better, so I will in a more appropriately titled mail 
message. :)
--
Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Object semantics

[Dan Sugalski]

Okay, I've been working on getting bytecode generation nailed down, 
and then digging into the vtable split stuff, but I thought I'd 
specify at least a little how objects (this would be real 
language-level "we're object-oriented dammit!" objects, not the 
lower-level stuff we're currently working with) should/will behave.

Parrot, like all the OO languages that will live on top of it, uses 
reference-style objects and non-reference values. The difference here 
is that on assignment values are cloned and references are copied. If 
you do this:

	a = b

and b is a value, a will have a copy of the value in b, but it'll be 
in a different location in memory, and changes to b won't affect a.

On the other hand, if b is an object, then a and be refer to the same 
object, and actions on either a or b affect the other. (This is nasty 
action at a distance, and I really don't like it, but nobody asked me 
here :)

We are going to muddle things some by having value objects in 
addition to reference objects, as arguably all the values that are in 
PMCs can be treated as value objects. (When those PMCs hold 
references to reference objects, the copied value is just an address 
so the destination object, and thus our value objects handle the case 
of being the front half of a reference object just fine)

Still, for 'real' OO, it's all reference object stuff, so things 
should Just Work. The pending questions about ruby assignment work 
out since, while the assignment part of vtable behaviour's 
potentially troublesome, it doesn't make any difference since what 
you're assigning into is the front half of a reference object, and 
thus you get pointer-to-pointer semantics the way you want. (And if 
it's not, then it should do the right thing on assignment, which may 
be... interesting in a mixed-language environment)
--
Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

GC/DOD feedback & runtime tuning

[Dan Sugalski]

I've been playing with a few test programs, and I'm finding it 
depressingly easy to get perl 5 to beat the pants off of Parrot for 
runtime speed, often by a factor of 4, regularly by a factor of two. 
There are a number of trouble spots that we can address now, without 
affecting anything else that's going on.

First, the resource system falls down hard when doing lots of 
allocations with no deallocations. It gets exponentially slower, 
which is a Bad Thing. While I'm not sure that there's a whole lot to 
be done about this in the grand scheme, the current system can 
certainly use some feedback mechanisms to deal with the problem for 
reasonable (<100K live objects) data set sizes.

Second, Array and its subclasses need some performance thumping, as 
they seem to come in at about half the speed of perl 5. I'm not sure 
where the speed penalty comes in, whether it's in the class code or 
in the design of the access mechanisms, but I want to find out why, 
and fix it *now*. If it's a design problem, I want to know so we can 
redesign before too much is dependent on the current scheme, and if 
it's an implementation tuning issue that can be dealt with by anyone 
with some time and an inclination to abuse the code.

I'll check in the stress programs and their perl 5 counterparts so we 
can have some sort of feel for time issues on this.
--
Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: Infant mortality

[Leopold Toetsch]

Nicholas Clark wrote:


On Thu, Jan 02, 2003 at 08:12:45PM +0100, Leopold Toetsch wrote:



[ refcounting ]



Effectively everything currently holding an object needing active
destruction becomes an object needing active destruction, (for the duration,
but no longer), and so on all the way back up to the root set.

Whether in practical terms this turns out to mean virtually everything
anyway, I don't know.



As you don't know, where such an object may be stored, you have the 
"virtually everything".


We're not guaranteeing perfection, though, are we? Just better than perl5?



Dan did outline our strategy, which seems to be very reasonbale for me. 
If timers or such have to be destroyed at the end of scope, the HL is 
responsible for telling parrot about this, either by an explicit "sweep" 
call, or - with more coding on parrots side - by some counting of "must 
destroy objects" + a full DOD run.


[ie things wanting timely destruction get their destructor called at the
correct time, unless you've been silly enough to send them off into a
refcount loop. In which case they will get spotted at some point probably
before program exit, compared with perl5 which only spots them if your
program exits]



Yep. A reference to an object is a reference the programmer of such code 
has to be aware of. And - Parrot doesn't suffer of refcount loops.


Nicholas Clark


leo

Re: Infant mortality

[bks24]

Here's a proposal w.r.t. adaptive GC (it's a variant of
the NEW/BABY/DONTGCME/whatever flag): all buffers are
created with this flag, but we only clear the flag on
DOD runs when we _know_ it's safe (e.g. between opcodes);
if we find we're getting low on resources, we queue a do-DOD-now
event to be run next time we check for pending events. The
plus is that there are no "what if we do DOD twice before
this object gets rooted" issues; the minus is that we
either have to fail allocations prematurely or give up
on guarranteed resource limits. On the other hand, if
someone _knows_ they're generating a lot of garbage in
a loop (in C, not Parrot), they are always free to do
a DOD run (if needed) when they know it's safe.

-- BKS

Re: Object semantics

[Leopold Toetsch]

Dan Sugalski wrote:



Still, for 'real' OO, it's all reference object stuff, so things should 
Just Work. The pending questions about 


... strings still is pending ;-)
Remember string_set ins substr and 50% live performance increase.

Do we want do reuse string headers like PMCs (and change semantics 
slightly) or do we not.

And: Subject "GC quick note"

One thing that might help the GC situation some.

Since we've got mutable strings (yep, it's OK) perhaps we should
thump the string functions a bit to not allocate new strings quite so
often. That'll leave us fewer headers to worry about.
--
 Dan

leo

Re: GC/DOD feedback & runtime tuning

[Leopold Toetsch]

Dan Sugalski wrote:



First, the resource system falls down hard when doing lots of 
allocations with no deallocations. It gets exponentially slower, which 
is a Bad Thing. 


Your are sure, that you didn't start swapping?
I have here this test pushing X integers into a PerlArray, then clone 
it, then do a sweep:

1e5
build 0.181974 seconds.
clone 0.056985 seconds.
sweep 0.082553 seconds.

5e5
build 0.834865 seconds.
clone 0.277557 seconds.
sweep 0.389210 seconds.

1e6 first time

build 1.305464 seconds.
clone 11.202339 seconds.
sweep 30.276119 seconds.


1e6 3rd time

build 1.293441 seconds.
clone 1.267864 seconds.
sweep 0.976740 seconds. 


I see a linear increse but obscure decrease for the first 1e6 test where 
 it seems to be that swapping occurs first time (i386/linux, 800 Mhz 
Athlon, 256 Meg Ram).


Second, Array and its subclasses need some performance thumping, as they 
seem to come in at about half the speed of perl 5. 


Ha, you didn't compare it before my list checkin/array rewrite ;-)



I'll check in the stress programs and their perl 5 counterparts so we 
can have some sort of feel for time issues on this.

When those are in, I'll have a look at it. There are some things in 
list.c that can be optimized. Did you compare mem usage of:

 new P0, .PerlArray
 set P0[100], 42

with perl5 too :-)

leo

Re: PRE / POST in loops

[Luke Palmer]

> Mailing-List: contact [EMAIL PROTECTED]; run by ezmlm
> Date: Thu, 2 Jan 2003 19:21:04 +0100
> Cc: [EMAIL PROTECTED]
> From: Arthur Bergman <[EMAIL PROTECTED]>
> X-SMTPD: qpsmtpd/0.20, http://develooper.com/code/qpsmtpd/
> 
> Hello,
> 
> I just got a question from Lee Pumphret regarding Hook::Scope POST in 
> loops.
> 
> Currently it treats every iteration as a scope entry and scope exit so.
> 
> for(1..3) {
>   POST { print 2 };
>   print 1;
> }
> 
> will print "121212",
> 
> Since perl6 seems to have a NEXT {} block for doing this, how is POST 
> and CATCH supposed to
> be used in conjunction with loop scopes.

The difference between POST and NEXT is simply that POST fails to
refrain from executing after the final iteration, while NEXT does not.

You example is correct.  In addition,

for 1..3 {
  print 1;
  NEXT { print 2 }
}

Prints "12121".

NEXT is really a kind of PRE block, contrary to what the example may
lead you to believe.  So, 

for 1..3 {
  print;
  NEXT { print }
}

Prints "12233", not "11223".

As I recall, CATCH can be used inside any block that contains an
expression that may throw an exception.

for 1..Inf {
die "Bonkers";
CATCH {
  print;
}
}

Prints "Bonkers".

I hope this clarified things.

Luke

Re: LXR or GLOBAL

[Robert Spier]

Mitchell N Charity wrote:
>Do you have a recommendation between GLOBAL and LXR?
> I'd suggest LXR.  It understands perl files out-of-the-box, is more

I've asked one of our volunteers who already runs some similar systems
for us if he can get this running.  If not, I'll see about setting you
(Mitchell) up with the resources you need.

If nobody hears back in a week or two on this topic, please ping me
again.

-R

Re: Infant mortality

[Dan Sugalski]

At 6:52 PM -0500 1/2/03, [EMAIL PROTECTED] wrote:

Here's a proposal w.r.t. adaptive GC (it's a variant of
the NEW/BABY/DONTGCME/whatever flag): all buffers are
created with this flag, but we only clear the flag on
DOD runs when we _know_ it's safe (e.g. between opcodes);
if we find we're getting low on resources, we queue a do-DOD-now
event to be run next time we check for pending events. The
plus is that there are no "what if we do DOD twice before
this object gets rooted" issues; the minus is that we
either have to fail allocations prematurely or give up
on guarranteed resource limits. On the other hand, if
someone _knows_ they're generating a lot of garbage in
a loop (in C, not Parrot), they are always free to do
a DOD run (if needed) when they know it's safe.


This doesn't work well at all with recursive calls into an 
interpreter, which is an issue. It also has some potentially large 
overhead as we need to do stuff between every opcode, whien generally 
we're going to be triggering runs when we're in the middle of an 
opcode.
--
Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: GC/DOD feedback & runtime tuning

[Dan Sugalski]

At 12:52 AM +0100 1/3/03, Leopold Toetsch wrote:

Dan Sugalski wrote:


First, the resource system falls down hard when doing lots of 
allocations with no deallocations. It gets exponentially slower, 
which is a Bad Thing.


Your are sure, that you didn't start swapping?


Oh, absolutely. One of the joys of a laptop, it's pretty obvious when 
the hard drive spins up. examples/benchmarks/stress.pasm sort of 
shows the problem. I managed to wipe an earlier version, but it was 
allocating arrays of arrays of 10K elements. It starts getting nasty 
reasonably quickly. I checked in examples/benchmarks/stress3.pasm to 
show what's going on. Interestingly, it triggers between 1 and 2 
sweeps per 10K elements allocated, which is odd as the interpinfo 
doesn't seem to show that we're actually out of anything. Either the 
counters are wrong or we're triggering DODs when we're not out of 
things, but whichever, we need to fix something.

The ever-increasing (by a factor of 4, it looks like) allocation size 
needs a cap as well--after a certain point, increasing the header 
allocation size is a bit much.

I see a linear increse but obscure decrease for the first 1e6 test 
where  it seems to be that swapping occurs first time (i386/linux, 
800 Mhz Athlon, 256 Meg Ram).


Second, Array and its subclasses need some performance thumping, as 
they seem to come in at about half the speed of perl 5.


Ha, you didn't compare it before my list checkin/array rewrite ;-)


Oh, I saw them, and they were pathetic. Much less pathetic now, which 
is darned good. :)

Still, we can (and have to) do better. I figure it's a good place for 
someone who's looking to get into the source to fiddle with. It's 
reasonably small and self-contained.

I'll check in the stress programs and their perl 5 counterparts so 
we can have some sort of feel for time issues on this.

When those are in, I'll have a look at it. There are some things in 
list.c that can be optimized. Did you compare mem usage of:

 new P0, .PerlArray
 set P0[100], 42

with perl5 too :-)

Check out the stress tests, but not quite that extreme. :)
--
Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk