Re: Meta-design

[Nicholas Clark]

On Wed, Dec 06, 2000 at 02:05:32AM +, Simon Cozens wrote:
> decent implementations, and we don't have enough experience of it. C's
> portable and everyone knows it, but it's a swine for doing OO things.

but perl5 is *still* encountering platforms that don't fully meet the 1989
ANSI spec for parts of the libraries, so anyone who would like to rely on
C99 for any part of the core is welcome to spend the next 10 years working
around portability problems. :-(

[I'm thinking fflush(NULL) here.
 It might be interesting to have -V print out all the non-ANSI thinks that
 perl5 Configure found (for our information on user platforms)
 (void flags, volatile, const, headers absent, functions absent)
]

Nicholas Clark

Re: Meta-design

[Nicholas Clark]

On Tue, Dec 05, 2000 at 11:12:01PM -0500, Bradley M. Kuhn wrote:
> helps people think about the issues.  However, I would like it to be
> possible for a programmer of language Foobar to take the specification of
> the data structures and implement them directly in Foobar without too much
> trouble.  The fewer C-isms in the high-level spec, the easier such a task
> will be.

I agree
 
> As the RFC 125 discussed, I think the interfaces for data structures and
> other APIs to be somewhat object-oriented.  Basically, something like:
> 
>An SV must answer the following methods:
> 
>   UnicodeString toString()  # returns the scalar value as Unicode string
>   ASCIIString   toString()  # returns the scalar value as an ASCII string
>   int32 toInt() # returns the scalar value as a 32-bit int.
>
> 
> or something like that.  

2 things the above reminds me to watch out for

   int32 toInt32()

don't assume anything about integer sizes (or any machine native type sizes
or precisions, or that floating point can (or can't) preserve integers)

  UnicodeString toString()  # returns the scalar value as Unicode string
  ASCIIString   toString()  # returns the scalar value as an ASCII string

C++ implementors will have fun with names that assume one can overload on
return type.


These won't be the only things to be wary of. Presumably a list of such
guidelines are something useful for someone to maintain on the meta-design
list. (Is maintaining such a document an "apprentice" job? (see perl6-meta))

Nicholas Clark

Re: Meta-design

[Jarkko Hietaniemi]

> No, there isn't. I object to targetting GCC specifically for two reasons, 
> though, neither of them VMS related:
> 
> 1) Targeting a single compiler, no matter whose it is, is a bad idea. We're 
> writing in a *language*, not for a compiler. Targeting a specific compiler 
> restricts us even more than choosing a language.

Strongly agreed.

> 2) GCC produces slow code on all platforms where there's an alternative. 
> Compaq C beats it on Alphas and VAXen, Sun's compiler beats it on SPARC 
> machines, and HP's beats it on PA-RISC machines. Heck, Microsoft's compiler 
> beats it on x86 chips. (As does Intel's compiler) We want perl fast, and 
> crippling it by requiring a particular compiler's a foolish thing.

Yea, truly.

-- 
$jhi++; # http://www.iki.fi/jhi/
# There is this special biologist word we use for 'stable'.
# It is 'dead'. -- Jack Cohen

Re: Meta-design

[Jarkko Hietaniemi]

> This example shows that you can offend all
> "religious" camps and still capitalize on their 
> disparate arguments.

Sounds like keeping the holidays of all the religions and but none of
the penances.  I like it :-)

-- 
$jhi++; # http://www.iki.fi/jhi/
# There is this special biologist word we use for 'stable'.
# It is 'dead'. -- Jack Cohen

RE: Meta-design

[Evan Howarth]

> Decisions of GC models rapidly become religious arguments.

The most important argument is: Does a particular 
GC model meet the Perl 6 language requirements?

Reference counting fulfills the language requirement 
for deterministic destructor calls. (Those 
destructors might be freeing non-memory resources 
such as Oracle database connections and file 
handles.) However, it forces language users to 
beware of circular references. In the case of circular 
references, destructors are not called and memory not
freed until the program ends.

Mark and sweep GC solves the circular reference
problem. Also, it can sometimes provide faster 
execution. However, it loses deterministic destructor
calls. Expensive or rare resources are not freed as 
soon as possible with mark and sweep schemes.

Perhaps a hybrid approach would solve all the
language needs. Retain reference counting. When the
count drops to zero, call the destructor and mark 
the object. Free marked objects when sweep runs.
To reclaim circular references, the sweep task may
periodically scan the stack, globals, etc. The scan
would find circular references and set their counts 
to zero. Because circular references are rare, the 
scan need not occur every time a sweep is performed.
This example shows that you can offend all
"religious" camps and still capitalize on their 
disparate arguments.

The language requirements should drive the selection
of a GC model -- not fear or deference to a set of
idealistic opinions.

Re: Meta-design

[Nathan Torkington]

Simon Cozens writes:
> This is not a design document; it's a meta-design document - that is, it
> tells us what things we need to design, the things we need to consider
> during the design process of the Perl 6 internals.

Thank you!

> Why does string C have to screw everything up?

It doesn't.  String eval is the escape hatch from a language that
can't do what you want it to do.  As such it's okay for it to be
slow--consider it incentive to fix the language :-)  The runtime
system should be able to load the compilation system and say
"turn this SV into a CV in this lexical scope," and then call the
CV.

> Today's scary thought:
> 
> use B; my $main_root = new B::OP;
> ...
> my $int = new B::Interpreter; $int->run($main_root)

That's yesterday's scary thought.  Today's scary thought is making
a perl5 XS wrapper for the perl6 component libraries and being able
to run perl6 from within perl5 in that way.

Nat

Re: Meta-design

[Daniel Chetlin]

On Wed, Dec 06, 2000 at 08:31:07AM -0700, Nathan Torkington wrote:
> Simon Cozens writes:
> > Why does string C have to screw everything up?
> 
> It doesn't.  String eval is the escape hatch from a language that
> can't do what you want it to do.  As such it's okay for it to be
> slow--consider it incentive to fix the language :-)  The runtime
> system should be able to load the compilation system and say
> "turn this SV into a CV in this lexical scope," and then call the
> CV.

The problem isn't the speed of `eval STRING' or the compilation system,
it's how to make it exist and work. To make `eval STRING' work, the
whole of the compiler has to be there, waiting, whenever a program is
run. That means a theoretical binary compile of `print "Hello world\n"'
has to come packaged with howevermany megs the Perl executable takes up.
And that all ports, for example JVM and Palm, have to implement the
compiler natively, rather than just having code compiled elsewhere and
sync-ed to the Palm or run from the JVM.

Simply deciding that `eval STRING' is "unimplemented" on these
theoretical ports and binary compiles is the best idea I've heard yet,
but we should remember that `require' is built on `eval STRING'.

-dlc

Re: Meta-design

[Dan Sugalski]

At 07:49 AM 12/6/00 -0800, Daniel Chetlin wrote:
>Simply deciding that `eval STRING' is "unimplemented" on these
>theoretical ports and binary compiles is the best idea I've heard yet,
>but we should remember that `require' is built on `eval STRING'.

That's not necessarily the case. If we allow for pre-compiled modules (and 
that's one of the design goals) then require (and do) can still function 
with those. A palm might not be able to load in foo.pl, but it could load 
in foo.plc.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: Meta-design

[Dan Sugalski]

At 08:31 AM 12/6/00 -0700, Nathan Torkington wrote:
>Simon Cozens writes:
> > Today's scary thought:
> >
> > use B; my $main_root = new B::OP;
> > ...
> > my $int = new B::Interpreter; $int->run($main_root)
>
>That's yesterday's scary thought.  Today's scary thought is making
>a perl5 XS wrapper for the perl6 component libraries and being able
>to run perl6 from within perl5 in that way.

Nah, that's simple, as long as we get the namespaces sorted out.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

A moment about software quality...

[Dan Sugalski]

http://www.salon.com/tech/feature/2000/12/06/bad_computers/index.html

And no, open source software is *not* immune.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: Tech documentation (Re: Perl Apprenticeship Program)

[Nathan Wiger]

Kirrily Skud Robert wrote:
> 
> Open Source Writers Group (http://oswg.org/) is a good starting point.
> I'm subscribed to their mailing list. 

This is really cool. Should we consider posting an announcement to this
website for potential docs people? Or is it still premature to do
something like that?

-Nate

Re: Tech documentation (Re: Perl Apprenticeship Program)

[David Grove]

 > >Open Source Writers Group (http://oswg.org/) is a good starting point.
 > >I'm subscribed to their mailing list.  I can think of a couple of
other
 > >good places to try, too, but they're a bit politically incorrect to
 > >mention in this context :-/
 >
 > Who on earth would be considered politically incorrect in this context?

Nobody hit me, but I just figured he meant a sex stories writers guild or
something and figured it was a wry joke. (If I'm wrong I got a good giggle
anyways and didn't miss the point.)

p

Re: Meta-design

[Simon Cozens]

On Wed, Dec 06, 2000 at 08:31:07AM -0700, Nathan Torkington wrote:
> > Today's scary thought:
> > 
> > use B; my $main_root = new B::OP;
> > ...
> > my $int = new B::Interpreter; $int->run($main_root)
> 
> That's yesterday's scary thought.  Today's scary thought is making
> a perl5 XS wrapper for the perl6 component libraries and being able
> to run perl6 from within perl5 in that way.

No, today's scary thought is that I'm half way to implementing the above.

-- 
For detailed information on the "info" command, type "man info".
- plan9 has a bad day

Re: OO AV/HV's and tie (was Re: Meta-design)

[Simon Cozens]

Oh boy, it's OO syntax nargery time again. *sigh*.

On Wed, Dec 06, 2000 at 10:51:14AM -0800, Nathan Wiger wrote:
>@array->length
>%hash->keys
> 
> Simply keeping @arrays and %hashes as buckets for SV's wouldn't let you
> do this.

I don't think that's true. At all.

> An "SV" would really just be an AV with
> one value. HV's would remain separate since there's more to them (direct
> access, etc) than just multiple values. Again, just something worth
> tossing around. 

I'm in favour of the exact opposite: an AV is "just" an SV-alike vtable
with array methods instead of scalar methods and a pointer to some
storage, (probably an array of SVs) and likewise an HV. That would allow
(array->length)() which seems to be what you want above.

> I think it would be cool

Good for you. This is internals design; perl6-language is over there ---> 
and the "ph33r mY |<" phase is supposed to be over now 
anyway.

> It's probably worth discussing the "typeglobs must die" issue here as
> well (even though I like them :-).

No, it's probably not.

-- 
"Having just ordered 40 books and discovered I have no change out of a grand, 
I'm thinking of getting a posse together and going after some publishers. I'd 
walk into a petrol station and buy lots of petrol on Monday, too, but I think 
I'd get funny looks. More funny looks." - Mark Dickerson

Re: OO AV/HV's and tie (was Re: Meta-design)

[Simon Cozens]

On Wed, Dec 06, 2000 at 10:51:14AM -0800, Nathan Wiger wrote:
> Don't forget we can mix-and-match C/C++ to some degree

for added portability!

-- 
If computer science was a science, computer "scientists" would study what 
computer systems do and draw well-reasoned conclusions from it, instead of
being rabid clueless wankers who've never even seen a real world system before,
let alone used one. These are the kind of people that brought us pascal, folks.
 - Charles J Radder, asr.

What's a PDD for the rest of us?

[Bryan C. Warnock]

In http://www.mail-archive.com/perl6-internals@perl.org/msg01766.html,
Dan outlined rev 2 of the RFC process, PDD (Perl Design Docs).

Everything seems rather specific to internals - will there be a separate
mechanism in place for all-things non-internal, say, for instance, the format
for a non-internal mechanism mirroring the PDD?

We seem to be discussing some meta-changes, but we don't currently have a
sanctioned method of presenting a formal (continuously modified) picture of
what is current for topic X in Perl 6.

And speaking of current, out of curiosity, where are we in regards to Larry's
RFC review?  I certainly appreciate the non-rush, but it's been quiet.

Almost *too* quiet


  -- 
Bryan C. Warnock
RABA Technologies

Re: What's a PDD for the rest of us?

[Dan Sugalski]

At 02:39 PM 12/6/00 -0500, Bryan C. Warnock wrote:
>In http://www.mail-archive.com/perl6-internals@perl.org/msg01766.html,
>Dan outlined rev 2 of the RFC process, PDD (Perl Design Docs).
>
>Everything seems rather specific to internals - will there be a separate
>mechanism in place for all-things non-internal, say, for instance, the format
>for a non-internal mechanism mirroring the PDD?

Well, until Larry releases the spec, just about everything else is frozen, 
though we can certainly expand the PDD stuff to include the standard 
library, documentation, and QA stuff. (It seems a tad premature to be 
proposing language changes when we neither have a final language spec nor 
an implementation of that spec...) PDDs don't have to be all internal--I 
changed the name to deal with the confusion with the IETF's RFCs. (We were 
fast creeping up on the active RFC range)

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

RE: Meta-design

[Dan Sugalski]

At 06:47 AM 12/6/00 -0800, Evan Howarth wrote:
>Perhaps a hybrid approach would solve all the language needs. Retain 
>reference counting. When the count drops to zero, call the destructor and 
>mark the object. Free marked objects when sweep runs. To reclaim circular 
>references, the sweep task may periodically scan the stack, globals, etc. 
>The scan would find circular references and set their counts to zero. 
>Because circular references are rare, the scan need not occur every time a 
>sweep is performed. This example shows that you can offend all "religious" 
>camps and still capitalize on their disparate arguments.

I'm looking to completely decouple destruction and garbage collection in 
perl 6. Garbage collection's really an internal thing, something that a 
perl programmer shouldn't need to worry about. It's housekeeping, and 
that's what perl's supposed to do automagically.

What I'm thinking is that we'll have a scoped destruct stack that gets 
pointers to variables that explicitly need destruction, and as we exit 
levels of scope we call the destructors of those variables that need it. 
(They can still be GC'd later to pick up their now-free memory) Most things 
won't get tossed on there, since most variables don't have any destruction 
behaviour.

This will hopefully be a performance win in some cases, since most of the 
reference works I've got indicate that reference counting GC is the most 
expensive, as you spend a lot of CPU time twiddling the ref count. (Though 
I do know that a number of the GC implementations get around this somewhat)

Decoupling GC and destruction also means we can change the GC when we think 
it's necessary--it may turn out that our initial choice is sub-optimal, and 
switching to another later buys us some performance, and that's keen too. 
(It also means we can start with a relatively naive GC, say a simple mark 
and sweep, or copying collector, and tune it later once everything else is 
working)

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: Meta-design

[Nicholas Clark]

On Wed, Dec 06, 2000 at 01:15:07PM -0500, Dan Sugalski wrote:
> What I'm thinking is that we'll have a scoped destruct stack that gets 
> pointers to variables that explicitly need destruction, and as we exit 
> levels of scope we call the destructors of those variables that need it. 
> (They can still be GC'd later to pick up their now-free memory) Most things 
> won't get tossed on there, since most variables don't have any destruction 
> behaviour.

The one specific case that someone (sorry, forget who, not kept that e-mail)
mentioned was using objects to hold "expensive" things such as database
handles that you want closed as soon as the last referent stops using them
("Would the last person leaving please turn out the lights")
Stack based destructors as you described them would fire as soon as any
reference went out scope?
[hmm. am I thinking too much like perl5 here]
sort of explicit call to DESTROY, akin to anyone walking out the door
turning the lights out without checking if someone else is still there.

and just leaving it to the GC would ensure that the lights remained on
while people were in Seattle^Wthe room, but might mean that they stayed
on for quite a while after the last person left.

However, for the case of a shared object wanting the refcount behaviour,
would it be easily implemented with the primitives you describe?
[scope stack, lazy garbage collection] for example a module using one
hub object with (perl) methods to maintain a refcount, and a second class
of spoke object with copied instances sitting on each scope stack to
decrease the hubs reference count at the correct time.

(ie am I right in thinking we can implement a solution to "Would the last
person leaving please turn out the lights" without needing reference counts
in the core SV)

Nicholas Clack

OO AV/HV's and tie (was Re: Meta-design)

[Nathan Wiger]

Simon Cozens wrote:
> 
> =head2 Implementation Language
> 
> C++ gives us OO and headaches, is wildly non-portable due to a lack of
> decent implementations, and we don't have enough experience of it. C's
> portable and everyone knows it, but it's a swine for doing OO things.
 
Don't forget we can mix-and-match C/C++ to some degree, so it's not
necessarily an either/or choice.

> =head2 Variable Storage
> 
> AVs - can we do anything better than an array of pointers?
>
> HVs - a hash is a hash is a hash, so fundamentally there's little
> we can do with HVs.

One caveat is something that's been discussed somewhat, and that is the
ability to do this:

   @array->length
   %hash->keys

Simply keeping @arrays and %hashes as buckets for SV's wouldn't let you
do this. Conversely, the overhead required to maintain SV-like AV/HV's
may kill any potential benefits. Regardless, if we want that
functionality it could impact the design considerably.

This is perhaps one worthwhile argument for at least combining arrays
and scalars into a single "highlander" type. No, I'm not a wholehearted
supporter of that position by any means, but it would solve lots and
lots of problems with context. An "SV" would really just be an AV with
one value. HV's would remain separate since there's more to them (direct
access, etc) than just multiple values. Again, just something worth
tossing around. 

> Let's also not forget that we need to provide a method of allowing
> user-defined data types. How does this fit in with the vtable model? Do
> we expect people to provide functions to fit a variable API - if so, we
> need to define that API - or do we allow inheritance from a base
> data type? Is this the sort of thing that's likely to replace ties?

I would hope so. :-) This brings up the whole hairy issue of how
fundamental we can make overloading and inheritance. For example, I
think it would be cool if all stringification was simply:

   $some_sv_thingy->STRING;

And the CORE/UNIVERSAL/whatever STRING was just:

   sub STRING { return $self->VALUE }

or something like that. So, rather than having to make decisions and
look for methods (like the "magic" - pardon the pun - of tie currently),
STRING could just be called, whatever that STRING was. The same concept
can be applied to ADD for addition, etc. Time permitting, I've been
digging through Python and there are some things Perl could potentially
steal in this area. 

> =head2 Symbol Table Handler
> 
> One thing that needs careful thought is the way Perl-side variables are
> stored; global variables can be easily dealt with by putting them into a
> hash, which is exactly what happens right now. Lexical variables are
> more tricky, and dynamic variables with C make the situation very
> difficult indeed. Is there a better symbol handling model for lexical
> and dynamic variables?

It's probably worth discussing the "typeglobs must die" issue here as
well (even though I like them :-).

-Nate

Call for apprentice: was Re: Meta-design

[Simon Cozens]

On Wed, Dec 06, 2000 at 11:48:45AM +, Nicholas Clark wrote:
> (Is maintaining such a document an "apprentice" job? (see perl6-meta))
 
I'd certainly like someone to take on the maintainance of the document I
posted last night, because I hope that one of its functions will be to explain
to apprentices a little of some of the concepts that we're dealing with; I
tried to do that in places, but it got late. 

So, anyone want to volunteer? 

What maintainance would involve would be, I think, receiving forwarded
messages from perl6-internals-design (or whatever) with bits marked "We need
to remember to think about this more!", and making sure those bits are covered
on the meta-design, with an option on learning about the topic and writing a
short summary for everyone to gain from. It would probably also need updating
when we settle one of the issues on it, with a brief note on what was decided.

Simon

-- 
Putting heated bricks close to the news.admin.net-abuse.* groups.
-- Megahal (trained on asr), 1998-11-06

Re: Tech documentation (Re: Perl Apprenticeship Program)

[David Grove]

Kirrily Skud Robert <[EMAIL PROTECTED]> wrote:

 > On Tue, Dec 05, 2000 at 11:28:31AM -0800, Nathan Wiger wrote:
 > >
 > > Anyways, that's just one suggestion. Do I have any idea where to find
 > > these mythical people? No, unfortunately. Perhaps some feelers on
 > > newsgroups might be a good place to start. Personal experience shows
 > > that this could be a win, though.
 >
 > Open Source Writers Group (http://oswg.org/) is a good starting point.
 > I'm subscribed to their mailing list.  I can think of a couple of other
 > good places to try, too, but they're a bit politically incorrect to
 > mention in this context :-/
 >
 > K.

Apologies to this author. I didn't know he was a she, and I mean no
offense at my guess at what group was politically incorrect. It just
looked like Dan had seen the link and missed a bit later on.

Of course realizing that documentation is good no matter where it comes
from, I think we need to scour around on the inside as much as possible
before looking to "outsource". Perl writing has a sort of unique humor
(and odd linguistics "perlisms"), which has helped to form and identify
the culture. I think it would be a shame to lose that if we pawn off too
much onto people not "into" the culture.

I won't discount her suggestion though. Any documentation is good
documentation, especially on behalf of programmers.

Re: What's a PDD for the rest of us?

[Bryan C. Warnock]

On Wed, 06 Dec 2000, Dan Sugalski wrote:
> Well, until Larry releases the spec, just about everything else is frozen, 
> though we can certainly expand the PDD stuff to include the standard 
> library, documentation, and QA stuff. (It seems a tad premature to be 
> proposing language changes when we neither have a final language spec nor 
> an implementation of that spec...) PDDs don't have to be all internal--I 
> changed the name to deal with the confusion with the IETF's RFCs. (We were 
> fast creeping up on the active RFC range)

Understood.  One other quick question.  Are the PDDs expect to go terminal at
the delivery of Perl 6, or should the active PDDs continue to evolve and govern
Perl 6 maintenance as well?  (If that makes any sense.)

 -- 
Bryan C. Warnock
RABA Technologies

Re: What's a PDD for the rest of us?

[Dan Sugalski]

At 03:08 PM 12/6/00 -0500, Bryan C. Warnock wrote:
>On Wed, 06 Dec 2000, Dan Sugalski wrote:
> > Well, until Larry releases the spec, just about everything else is frozen,
> > though we can certainly expand the PDD stuff to include the standard
> > library, documentation, and QA stuff. (It seems a tad premature to be
> > proposing language changes when we neither have a final language spec nor
> > an implementation of that spec...) PDDs don't have to be all internal--I
> > changed the name to deal with the confusion with the IETF's RFCs. (We were
> > fast creeping up on the active RFC range)
>
>Understood.  One other quick question.  Are the PDDs expect to go terminal at
>the delivery of Perl 6, or should the active PDDs continue to evolve and 
>govern Perl 6 maintenance as well?  (If that makes any sense.)

I want them to continue, though whether they do depends on who has the 
magic sucker hat once things go final. It's my intent that anyone who wants 
to get complete documentation for X (where X could be the parser API, or 
the vtable API, or the guaranteed behaviour of the GC, or the PerlIO API 
and required behaviour) can grab the PDD covering it and be set.

Something, more or less, like the IETF standards RFCs, only with revisions 
of each PDD rather than a half zillion "supercedes or upgrades" RFCs.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: Perl Apprenticeship Program

[Graham Barr]

On Tue, Dec 05, 2000 at 10:00:50PM +, Nick Ing-Simmons wrote:
> B. The "master" / "apprentice" relationship is just that - it depends
>how the people in question relate. As a potential "master" I am all 
>too aware that I am not skilled in teaching - usually because I don't
>know what is obvious vs what is obscure - so anyone "taught" by me
>has to ask questions rather than be lectured to.

I can vouch for that, but you under estimate yourself :)

Graham.

RE: Meta-design

[Sam Tregar]

On Wed, 6 Dec 2000, Dan Sugalski wrote:

> >my $new_dog;
> >{
> >   my $dog = new Dog;
> >   $new_dog = \$dog;
> >}
>
> That would hoist the Dog reference into an outer level of scope--in this
> case the one containing $new_dog. Or so my thinking goes at the moment,
> though there may be (almost inevitably are) problems with that.

"hoist"?  I thought this was a stack you were talking about.  You're going
to do an O(n) operation on a stack every time a reference is passed out of
scope?  What happens when two references diverge:

   my $newDog1;
   {
 my $newDog2;
{
   my $dog = new Dog;
   $newDog2 = $newDog1 = \$dog;
}
 $newDog1 = undef;
}

How does a non-refcounting scheme know that $newDog2 is the last ref to
$dog when the first block is done?

As far as claims about mark-and-sweep improving performance, I guess we'd
need a test implementation in order to find out.  Considering the
non-deterministic character of many mark-and-sweep systems performance
testing can be a delicate matter.

-sam

Re: OO AV/HV's and tie (was Re: Meta-design)

[Nathan Wiger]

Simon Cozens wrote:
> 
> Oh boy, it's OO syntax nargery time again. *sigh*.

> > I think it would be cool
> 
> Good for you. This is internals design; perl6-language is over there --->
> and the "ph33r mY |<" phase is supposed to be over now
> anyway.

Cool! Thanks alot for the useful feedback. Comments like this are
certainly beneficial to the goal of Perl 6. Sorry for encroaching on
your god-like prowess of the subject matter. 

-Nate

RE: Meta-design

[Dan Sugalski]

At 03:38 PM 12/6/00 -0500, Sam Tregar wrote:
>On Wed, 6 Dec 2000, Dan Sugalski wrote:
>
> > >my $new_dog;
> > >{
> > >   my $dog = new Dog;
> > >   $new_dog = \$dog;
> > >}
> >
> > That would hoist the Dog reference into an outer level of scope--in this
> > case the one containing $new_dog. Or so my thinking goes at the moment,
> > though there may be (almost inevitably are) problems with that.
>
>"hoist"?  I thought this was a stack you were talking about.  You're going
>to do an O(n) operation on a stack every time a reference is passed out of
>scope?  What happens when two references diverge:

There are ways around that--the destruct stack could have a level of 
indirection in it, though that's getting rather more complex than I'd like.

>my $newDog1;
>{
>  my $newDog2;
> {
>my $dog = new Dog;
>$newDog2 = $newDog1 = \$dog;
> }
>  $newDog1 = undef;
> }
>
>How does a non-refcounting scheme know that $newDog2 is the last ref to
>$dog when the first block is done?

Hmmm. Okay, bad plan. I don't have a Plan B, though. We may want a 
refcounting scheme for destructed objects, or variables that get referenced.

>As far as claims about mark-and-sweep improving performance, I guess we'd
>need a test implementation in order to find out.  Considering the
>non-deterministic character of many mark-and-sweep systems performance
>testing can be a delicate matter.

All I have to go on is the literature, so this is somewhat shaky, but... 
Non-refcounting GC schemes are more expensive when they collect, but less 
expensive otherwise, and it apparently is a win for the non-refcount 
schemes. There are techniques to spread the GC cost around as well, so 
there aren't huge freezes when it comes time to collect the heap. If perl 
takes a more active stance with it's trash as well (where blocks can 
immediately toss variables that are guaranteed to be trash) we may also be 
able to lower the impact of GC.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

Re: OO AV/HV's and tie (was Re: Meta-design)

[Dan Sugalski]

At 12:48 PM 12/6/00 -0800, Nathan Wiger wrote:
> > Good for you. This is internals design; perl6-language is over there --->
> > and the "ph33r mY |<" phase is supposed to be over now
> > anyway.
>
>Cool! Thanks alot for the useful feedback. Comments like this are
>certainly beneficial to the goal of Perl 6. Sorry for encroaching on
>your god-like prowess of the subject matter.

Folks, I'd really rather this sort of crankiness not happen on the list 
(and yes, I mean everyone), and when it does please leave it be. Thanks.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

RE: Meta-design

[Sam Tregar]

On Wed, 6 Dec 2000, Dan Sugalski wrote:

> Non-refcounting GC schemes are more expensive when they collect, but less
> expensive otherwise, and it apparently is a win for the non-refcount
> schemes.

Which is why GC is intimately tied to DESTROY consideration in terms of
Perl.  If we intend to honor predictable DESTROY timing, and I think we
should, then we will need to reference count.  No ifs, elses or
alternations.  Anyone care to refute?

If we're going to be ref-counting anyway then the performance gain of a
non-refcounting GC, avoiding counting, is basically moot.  If we're
ref-counting for DESTROY timing then we may as well use that data in the
GC.

I'm not some kind of ref-count true-believer - if you think we should put
this discussion of to a later date then I'm cool.  I'm just spoiling for
some Perl 6 work to do and this area seemed ripe for critical development.

-sam

Re: OO AV/HV's and tie (was Re: Meta-design)

[Dan Sugalski]

At 07:55 PM 12/6/00 +, Simon Cozens wrote:
>Oh boy, it's OO syntax nargery time again. *sigh*.
>
>On Wed, Dec 06, 2000 at 10:51:14AM -0800, Nathan Wiger wrote:
> >@array->length
> >%hash->keys
> >
> > Simply keeping @arrays and %hashes as buckets for SV's wouldn't let you
> > do this.
>
>I don't think that's true. At all.

Definitely not.

> > An "SV" would really just be an AV with
> > one value. HV's would remain separate since there's more to them (direct
> > access, etc) than just multiple values. Again, just something worth
> > tossing around.
>
>I'm in favour of the exact opposite: an AV is "just" an SV-alike vtable
>with array methods instead of scalar methods and a pointer to some
>storage, (probably an array of SVs) and likewise an HV. That would allow
>(array->length)() which seems to be what you want above.

Currently the only difference between accessing an AV's member's value, an 
HV's member's value, or an SV's value is an extra argument to the vtable call.

print $foo[0];

is

   foo->get_string[UTF_8](ARRAY, 0);

while

   print $foo

is

   foo->get_string[UTF_8](SCALAR);

There doesn't seem to be any reason to split array/hash access into 
separate fetch and convert vtable calls. Bleah. (And the only reason for 
the first arg in there is because you can't do varargs with 0 args in C)

> > I think it would be cool
>
>Good for you. This is internals design; perl6-language is over there --->
>and the "ph33r mY |<" phase is supposed to be over now
>anyway.

I think we can skip that, though.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

RE: Meta-design

[Dan Sugalski]

At 03:54 PM 12/6/00 -0500, Sam Tregar wrote:
>On Wed, 6 Dec 2000, Dan Sugalski wrote:
>
> > Non-refcounting GC schemes are more expensive when they collect, but less
> > expensive otherwise, and it apparently is a win for the non-refcount
> > schemes.
>
>Which is why GC is intimately tied to DESTROY consideration in terms of
>Perl.  If we intend to honor predictable DESTROY timing, and I think we
>should, then we will need to reference count.  No ifs, elses or
>alternations.  Anyone care to refute?

Sure, but only objects. (or, to be really paranoid, things referred to) 
Nothing else needs refcounting. All the refcounting code can be isolated in 
the reference creation and deletion code, and we don't have to pay it 
otherwise.


Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

RE: Meta-design

[Buddha Buck]

At 03:54 PM 12-06-2000 -0500, Sam Tregar wrote:
>On Wed, 6 Dec 2000, Dan Sugalski wrote:
>
> > Non-refcounting GC schemes are more expensive when they collect, but less
> > expensive otherwise, and it apparently is a win for the non-refcount
> > schemes.
>
>Which is why GC is intimately tied to DESTROY consideration in terms of
>Perl.  If we intend to honor predictable DESTROY timing, and I think we
>should, then we will need to reference count.  No ifs, elses or
>alternations.  Anyone care to refute?

This is not a complete refutation, but...

It seems to me that there are three types of thingies[1] we are concerned 
about, conceptually:

A) Thingies with no DESTROY considerations, which don't need refcounts.
B) Thingies with DESTROY methods, but aren't timing-sensitive.  They can be 
destroyed anytime after they die.  These don't really need refcounts either.
C) Thingies with DESTROY methods which need to be DESTROYed as soon as they 
die.  These would seem to need refcounts.

I think that distinguishing between B and C is a syntax issue out of scope 
here.  Although B could be lumped with A if we could tell B and C apart, 
I'll assume that we must lump B and C together.

If we could refcount only C for destruction, and let the GC-of-your-choice 
handle the actuall memory reclaimation, then the expense of refcounting 
should only affect C thingies.  I am uncertain what the ratio of C thingies 
to A thingies is, so I can't judge how big a win it is.

Theoretically, a non-refcount GC should never find any C thingies that 
would have a refcount>0, so the non-refcount GC shouldn't have to worry 
about it.


>If we're going to be ref-counting anyway then the performance gain of a
>non-refcounting GC, avoiding counting, is basically moot.  If we're
>ref-counting for DESTROY timing then we may as well use that data in the
>GC.

But we only care about the ref-count for DESTROY timing.  If we can avoid 
counting for DESTROY timing insensitive thingies, we may still have a net 
performance gain.


>I'm not some kind of ref-count true-believer - if you think we should put
>this discussion of to a later date then I'm cool.  I'm just spoiling for
>some Perl 6 work to do and this area seemed ripe for critical development.
>
>-sam

RE: Meta-design

[Sam Tregar]

On Wed, 6 Dec 2000, Dan Sugalski wrote:

> Sure, but only objects. (or, to be really paranoid, things referred to)
> Nothing else needs refcounting. All the refcounting code can be isolated in
> the reference creation and deletion code, and we don't have to pay it
> otherwise.

Good point.  I hadn't considered the predominance of non-objects in Perl
programs.  I suppose that any system that allowed us to get away without
having to refcount every simple SV would be a pretty big win even if
blessed SVs still had to be counted.

-sam

Re: OO AV/HV's and tie (was Re: Meta-design)

[Dan Sugalski]

At 07:34 PM 12/6/00 +, Simon Cozens wrote:
>On Wed, Dec 06, 2000 at 10:51:14AM -0800, Nathan Wiger wrote:
> > Don't forget we can mix-and-match C/C++ to some degree
>
>for added portability!
>
>--
>If computer science was a science, computer "scientists" would study what
>computer systems do and draw well-reasoned conclusions from it, instead of
>being rabid clueless wankers who've never even seen a real world system 
>before,
>let alone used one. These are the kind of people that brought us pascal, 
>folks.
>  - Charles J Radder, asr.

So, Simon, is your .sig monster a sentient AI, or does it just have darned 
good timing? :)

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

RE: Meta-design

[Sam Tregar]

On Wed, 6 Dec 2000, Dan Sugalski wrote:

> What I'm thinking is that we'll have a scoped destruct stack that gets
> pointers to variables that explicitly need destruction, and as we exit
> levels of scope we call the destructors of those variables that need it.
> (They can still be GC'd later to pick up their now-free memory) Most things
> won't get tossed on there, since most variables don't have any destruction
> behaviour.

If you don't reference count how do you protect yourself from DESTROYing
objects that are still referenced:

   my $new_dog;
   {
  my $dog = new Dog;
  $new_dog = \$dog;
   }

Did $dog just get erroneously collected by your destruct stack?  No?
Without reference counting?

Frankly these hybrid GC schemes look more like the *worst* of both worlds
than best - all the predictable performance problems of reference counting
with the unpredictable performance problems of mark and sweep!

-sam

RE: Meta-design

[Dan Sugalski]

At 03:14 PM 12/6/00 -0500, Sam Tregar wrote:
>On Wed, 6 Dec 2000, Dan Sugalski wrote:
>
> > What I'm thinking is that we'll have a scoped destruct stack that gets
> > pointers to variables that explicitly need destruction, and as we exit
> > levels of scope we call the destructors of those variables that need it.
> > (They can still be GC'd later to pick up their now-free memory) Most things
> > won't get tossed on there, since most variables don't have any destruction
> > behaviour.
>
>If you don't reference count how do you protect yourself from DESTROYing
>objects that are still referenced:
>
>my $new_dog;
>{
>   my $dog = new Dog;
>   $new_dog = \$dog;
>}

That would hoist the Dog reference into an outer level of scope--in this 
case the one containing $new_dog. Or so my thinking goes at the moment, 
though there may be (almost inevitably are) problems with that.

>Did $dog just get erroneously collected by your destruct stack?  No?
>Without reference counting?
>
>Frankly these hybrid GC schemes look more like the *worst* of both worlds
>than best - all the predictable performance problems of reference counting
>with the unpredictable performance problems of mark and sweep!

Modern GC schemes are better than what you're probably thinking of with 
naive mark and sweep. Generational garbage collection can take care of a 
lot of that, as can some partitioning techniques. There's evidence in the 
literature that switching from reference counting to a more sophisticated 
GC scheme can also save us 5-10% in CPU overhead. (Though that's not a 
guarantee--LISPlike languages likely have different GC performance 
characteristics than perl does)

Object destruction, FWIW, is a separate problem from garbage collection, 
and the two only have to mildly overlap. The only reason they do so much in 
perl 5 is the destruction scheme piggy-backs on the GC scheme.

Dan

--"it's like this"---
Dan Sugalski  even samurai
[EMAIL PROTECTED] have teddy bears and even
  teddy bears get drunk

intercal

[Kirrily Skud Robert]

 other languages are great. they are a source of features to 
  steal^Wborrow.
 INTERCAL
 except maybe intercal

this was only an academic example (was Re: Meta-design)

[Bradley M. Kuhn]

Nicholas Clark <[EMAIL PROTECTED]> wrote:
> 
> 2 things the above reminds me to watch out for
> 
>int32 toInt32()
> 
> don't assume anything about integer sizes (or any machine native type sizes
> or precisions, or that floating point can (or can't) preserve integers)
> 
>   UnicodeString toString()  # returns the scalar value as Unicode string
>   ASCIIString   toString()  # returns the scalar value as an ASCII string
> 
> C++ implementors will have fun with names that assume one can overload on
> return type.

Sorry.  I didn't mean those should be methods Scalars have!   I was just
trying to show the kind of documentation I'd like to see.  I wasn't trying
to produce said documentation.  :)

-- 
Bradley M. Kuhn  -  http://www.ebb.org/bkuhn

 PGP signature

Re: Meta-design

[Bradley M. Kuhn]

> > Why does string C have to screw everything up?

Nathan Torkington <[EMAIL PROTECTED]> wrote:
 
> It doesn't.  String eval is the escape hatch from a language that can't do
> what you want it to do.  As such it's okay for it to be slow--consider it
> incentive to fix the language :-) The runtime system should be able to
> load the compilation system and say "turn this SV into a CV in this
> lexical scope," and then call the CV.

Whether or not it is slow is not my concern.  My concern is *can* we support
it on architectures that don't have a native C compiler (i.e., the JVM).

I think that's the first and foremost concern with eval($string) (and hence,
the parser).  Slow things can usually be made faster, but "can't get there
from here" is often hard to solve.

-- 
Bradley M. Kuhn  -  http://www.ebb.org/bkuhn

 PGP signature

Re: Meta-design

[Bradley M. Kuhn]

> At 07:49 AM 12/6/00 -0800, Daniel Chetlin wrote:
> >Simply deciding that `eval STRING' is "unimplemented" on these
> >theoretical ports and binary compiles is the best idea I've heard yet,
> >but we should remember that `require' is built on `eval STRING'.
 
I see no reason to ghettoize powerful non-C-based systems just because we
write the canonical perl6 implementation in Perl.

Soon, there will likely be JVM systems that can run eval($string) quickly
enough, but not if it is written in C (as there is no C->JVM compiler).


-- 
Bradley M. Kuhn  -  http://www.ebb.org/bkuhn

 PGP signature

Re: Markup wars (was Re: Proposal for groups)

[Russ Allbery]

Bennett Todd <[EMAIL PROTECTED]> writes:

> My own personal favourite for archival format would be to stick with POD
> until and unless we can cons up something even Plainer than POD. I've
> got this dream that someday we'll be able to take something --- perhaps
> based on Damian's Text::Autoformat --- and use it to parse purely plain
> ASCII text, formatted nicely for screen display, with no markup at all,
> and garnish it with markup allowing it to be automatically translated
> into nice sexy HTML, or SGML according to various other DTDs, or XML, or
> POD, or the man or mandoc troff macros, or LaTeX, or whatever.

I've fiddled with this before and can do text to HTML; the rest is just a
question of picking different backends and shouldn't be *too* hard.  All
the heuristics for parsing text are inherently fragile, but if you follow
a standard text formatting style, it works reasonably well.

-- 
Russ Allbery ([EMAIL PROTECTED]) 

Re: Markup wars (was Re: Proposal for groups)

[Jarkko Hietaniemi]

On Wed, Dec 06, 2000 at 03:59:32PM -0800, Russ Allbery wrote:
> Bennett Todd <[EMAIL PROTECTED]> writes:
> 
> > My own personal favourite for archival format would be to stick with POD
> > until and unless we can cons up something even Plainer than POD. I've
> > got this dream that someday we'll be able to take something --- perhaps
> > based on Damian's Text::Autoformat --- and use it to parse purely plain
> > ASCII text, formatted nicely for screen display, with no markup at all,
> > and garnish it with markup allowing it to be automatically translated
> > into nice sexy HTML, or SGML according to various other DTDs, or XML, or
> > POD, or the man or mandoc troff macros, or LaTeX, or whatever.
> 
> I've fiddled with this before and can do text to HTML; the rest is just a
> question of picking different backends and shouldn't be *too* hard.  All
> the heuristics for parsing text are inherently fragile, but if you follow
> a standard text formatting style, it works reasonably well.

For more ideas there is the PolyglotMan (nee RosettaMan),
http://polyglotman.sourceforge.net/

> -- 
> Russ Allbery ([EMAIL PROTECTED]) 

-- 
$jhi++; # http://www.iki.fi/jhi/
# There is this special biologist word we use for 'stable'.
# It is 'dead'. -- Jack Cohen

Perl6 in Java? (was Re: Meta-design)

[Bradley M. Kuhn]

Dan Sugalski <[EMAIL PROTECTED]> wrote:
 
> More importantly, what we're doing is outside Java's area of competence. 
> We're writing system-level code here. Java isn't a system-level programming 
> language. This isn't a bad thing, but it means it's an inappropriate 
> solution to the problem. We might as well write perl 6 in perl.

I think that Java is closer to system-level programming than Perl, and
certainly not as close as C.

However, I don't think we should dismiss it out of hand because people don't
do a lot of systems programming C.  some of the things we are going to build
for C (if that's what we pick), are already there automatically in Java,
such as:

   * garbage collection for internal data structures

   * vtables

And, it will make the barrier for entry for new internals hacker lower.

I just don't think we should dismiss Java out of hand because it isn't
traditionally used for systems-level programming.

 

> 1) Targeting a single compiler, no matter whose it is, is a bad
> idea. We're writing in a *language*, not for a compiler. Targeting a
> specific compiler restricts us even more than choosing a language.

I would note that if we write in Java, we aren't targeting a single
compiler, although at the moment, the only efficient compiler for Java might
be GCJ.

 
> 2) GCC produces slow code on all platforms where there's an alternative. 

This is likely going to change with the release of GCC 3.0, based on current
benchmarks of test releases of GCC 3.0.

-- 
Bradley M. Kuhn  -  http://www.ebb.org/bkuhn

 PGP signature

documenting interfaces (was Re: Meta-design)

[Bradley M. Kuhn]

Dan Sugalski <[EMAIL PROTECTED]> wrote:

> [vtable stuff snipped]
 
> I don't think it's too early to be dealing with the way variables are 
> implemented, at least at some level. 

I agree; I didn't mean for it to sound like I was saying otherwise.

> Also, just because we do vtables under the hood doesn't mean that any port
> of perl needs to do it that way--all that's required is to support the
> exposed interface, and the vtable stuff is definitely in the "hidden guts"
> realm.


That's really my point---the interface should be documented outside of the
implementation.  I was just concerned because much of the internals has been
C-specific thus far.


-- 
Bradley M. Kuhn  -  http://www.ebb.org/bkuhn

 PGP signature

Re: Meta-design

[Jarkko Hietaniemi]

On Wed, Dec 06, 2000 at 08:30:06PM -0500, Bradley M. Kuhn wrote:
> > At 07:49 AM 12/6/00 -0800, Daniel Chetlin wrote:
> > >Simply deciding that `eval STRING' is "unimplemented" on these
> > >theoretical ports and binary compiles is the best idea I've heard yet,
> > >but we should remember that `require' is built on `eval STRING'.
>  
> I see no reason to ghettoize powerful non-C-based systems just because we

Powerful?  Java?  Excuse me, I have must have picked the wrong reality.

> write the canonical perl6 implementation in Perl.

I'll repent when and if the time comes but now, in the real world,
platforms that have C vastly outnumber the platforms that are Java
only.  As far as I know the number of Java-only platforms is significant
only in the Sun marketing material.

> Soon, there will likely be JVM systems that can run eval($string) quickly
> enough, but not if it is written in C (as there is no C->JVM compiler).

I have seen a real Java implementation by Sun.  It was written in C.

> -- 
> Bradley M. Kuhn  -  http://www.ebb.org/bkuhn

-- 
$jhi++; # http://www.iki.fi/jhi/
# There is this special biologist word we use for 'stable'.
# It is 'dead'. -- Jack Cohen

Re: Perl6 in Java? (was Re: Meta-design)

[Sam Tregar]

On Wed, 6 Dec 2000, Bradley M. Kuhn wrote:

> And, it will make the barrier for entry for new internals hacker lower.

Really?  Do you honestly believe there are more Java programmers than C
programmers?  Particularily in the Perl development community!

> I would note that if we write in Java, we aren't targeting a single
> compiler, although at the moment, the only efficient compiler for Java might
> be GCJ.

It speaks volumes that the only efficient compiler is vaporware.  How fast
is it?  Who knows - it's not finished!  Does it have major flaws?
Probably - but just wait and see how fast it is!

Let's move on - C is our only real option.  It's portable enough (proven
by perl5), it's fast enough (or nothing is) and we already have in-house
experts to lead the implementation.  Are we serriously considering
anything else?  PIL?  If someone here has a real proposal for a
better-than-cpp macro language then let's hear it.  It's worth considering
in a purely abstract keep-an-open-mind sense.

-sam

Re: Perl Apprenticeship Program

[Dave Storrs]

On Tue, 5 Dec 2000, Steve Fink wrote:

> David Grove wrote:
> 
> > Also, as far as documentation goes, I think it _should_ be written by
> > apprentices, so that non-masters can understand it too. That's always been
> 
> Except it's a particular duty that nobody really likes to perform. Which 


Actually, I kinda like it.  If someone is willing to answer my
questions so that I understand what it is I'm documenting, I'm willing to
do a good chunk of the documentation.

I'm currently working on some docs on handling binary data in Perl
(with pack, unpack, vec, etc) that I intend to submit to perl.com.

Dave