Re: [RfC] vtable->dump
On Sep-08, Leopold Toetsch wrote: > Nicholas Clark <[EMAIL PROTECTED]> wrote: > > On Fri, Sep 05, 2003 at 09:34:04AM -0400, Dan Sugalski wrote: > >> On Fri, 5 Sep 2003, Peter Haworth wrote: > > >> > With the seen hash approach, I wouldn't expect the hash itself to take > >> > nearly as much space as the frozen structure; > >> > >> My experience tracing variable graphs in Perl 5 indicates otherwise. > > > I think it may be worth listening to Dan on this one. > > I don't know why Storable.xs is using a general HV for mapping an > address to an integer (more isn't needed to serialize all IMHO - Dan's > next_for_GC approach doesn't provide more info). Perl5 does convert the > address to a string and uses all weirdness of hv.c code. This isn't > necessary for this special kind of problem. > > Parrot has some DOD counters for possibly active PMCs. A simple and > probably fix-sized[1] hash based on such a count with just a mapping of > {&pmc => id} takes 12 bytes per entry on 32-bit architecures and its > really fast. It's late so I'm probably being an idiot, but is all that is needed (1) a unique id for every pmc address, and (2) whether or not the PMC has been visited yet? If so, why not use the address itself for the id (or some variant, like index of the pmc in its arena, added to/or-ed with some base id for the arena itself)? And for the flag, have arenas preallocate a bit vector of seen flags for all their PMCs? (Or if you have the arena base ids, then you can have a single bit vector for all arenas.) Or is there no simple way of going from a PMC address to its arena? Perhaps it's no win to pollute the cache with chunks of the bit vector instead of the actual referenced PMC flagpoles. But if that's an issue then I can't help but think that most of the time the seen bit vector would be quite sparse, so we could go to a multi-level scheme: use a conservative vector that can answer definitely not seen vs maybe seen, and then look at the actual PMC flagpole (or, again, a separate bit vector if it somehow helps multithreading) for the true answer. An eighth of a bit per entry ought to be small enough, no? I suppose I ought to pay more attention to the conversation (and the current code) before butting in with suggestions. I've a feeling I'm solving the wrong problem.
Re: Of AST and YAL
Michal Wallace <[EMAIL PROTECTED]> wrote: > On Sun, 7 Sep 2003, Leopold Toetsch wrote: >> I'm currently investigating the AST (abstract syntax tree) interface >> for Parrot. For getting a feeling, how this could look like, I've >> implemented (some parts) of Yet Another Language (YAL). > I like it. What is this written in? C or Perl or what? > If it's in perl (or python of course) I'd like to merge > in all the pirate code generation stuff. Its C only, sorry. > (My plan for this week was to do something very similar, > and try to get a simple lisplike language to integrate > with python) I'm thinking of a AST text interface for imcc/parrot too. The lispish AST-dump produced by YAL is handy for debugging and testing, and it can be parsed easily to reconstruct the AST again. > Sincerely, > > Michal J Wallace leo
Re: IMCC register assigment bug?
Steve Fink <[EMAIL PROTECTED]> wrote: > I'm getting my register stomped on in the following code: > .sub _bogus > set $P1, $P0[0] > find_lex $P2, "x" > newsub $P3, .Sub, _bogus > .pcc_begin prototyped > .arg $P1 > .pcc_call $P3 > after_call: > .pcc_end > .end > Notice that both $P1 and $P2 get equated with P16. These are partly used once. There is no indication where $P0 is coming from so the register is happily reused ($P1 shouldn't get globbered though). Do you have a more complete example? Above should probably be a .pcc_sub with the array as input .param. Anyway, register allocation and PCC isn't finished yet. I'm still waiting for the famous last words regarding return conventions. > Steve leo
Re: Parrot Z-machine
OK. I think I've learned enough about Parrot to respond to this email intelligently. --- Nicholas Clark <[EMAIL PROTECTED]> wrote: > > - Is it not being ported because of a lack of tuits, or because > > it's extremely hard? > > We'd need dynamic opcode loading because we don't want to have the > Z-machine specific opcodes compiled into every parrot (or every other > specialist set of opcodes) Right. What you're saying here is that I'll need to write Zmachine.ops, or some such. It will include all the Z-machine operations, which the bytecode will call. > We'd want dynamic bytecode conversion because we want parrot to be > able to directly load Z-code files, rather than having to first run > an external program to convert them. Right. The problem that I see with this is that Z-code "story" files have a very definite header format, which is almost but not quite entirely unlike Parrot bytecode. Just for example, the first few bytes are totally different, but are necessary for both languages. One way to get around this, as you say, is to convert them beforehand. For example, write a Parrot header, followed by bytecode whose first command is "skip the next few hundred bytes", then have the Parrot header, then set all the ops that would read the header to add an offset to the address. Or something. Yuck. On the other hand, I'm not quite sure what else you can do. > Both these features are wanted to seamlessly run any "alien" > bytecode, such as Python's bytecode, Java bytecode or .NET bytecode. > > However, I don't think that we'd need them in place to begin working > on a Z-code port. (We'd just have to arrange to link in any specialist > Z-code ops for a while, and to convert Z-code before loading it) Good point! And since I can steal all of the C code for the ops, well, we're almost done with project! [What should I do now?] > > I don't think you'd necessarily need to know actual PASM. I think > that the tricky part is thinking about how to map from a stack machine > to a register machine. I've no idea what academic (or other) > literature there is on this. The simplest way to run a stack based > language on parrot would be to ignore most of the registers and just > use parrot's stack. However, this isn't going to be efficient. So > researching/working out how to efficiently map stack operations to use > more than a couple of registers would be very useful for all sorts of > stack based languages. Getting started on that would probably be very > helpful - you don't need to actually write the implementation PASM if > you're able to describe what needs to a co-volunteer. OK, here's where I get very confused. What I started to understand is that if we're reading bytecode natively, then it's silly to translate to PASM, because that's going backwards: PASM just gets compiled to parrot bytecode. [Actually, it might be fun to disassemble Zcode (disassemblers exist) to Z assembly, then translate that to PASM+, i.e., PASM that calls Zcode ops in addition to the regular ops, but uses the Parrot registers et al., then compile and run that. But if I understand correctly, that's not what "native" means.] >From what I can understand, "native" means (if we ignore dynamic bytecode conversion issues) running directly from the bytecode, but writing new ops. That is, not using PASM at all. In that case, do I even have access to Parrot's stack? (Or do I just need to access all of it through C?) I guess maybe my mistake is in thinking of PASM as Parrot. Obviously, I need to do some more learning. -Amir __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com
[perl #23751] [PATCH] f should be if in genclass.pl
# New Ticket Created by Michael Scott # Please include the string: [perl #23751] # in the subject line of all future correspondence about this issue. # http://rt.perl.org/rt2/Ticket/Display.html?id=23751 > f should be if on line 41 of parrot/classes/genclass.pl. -- attachment 1 -- url: http://rt.perl.org/rt2/attach/64385/47154/b50cfc/genclass.patch genclass.patch Description: genclass.patch
[perl #23752] [BUG] op 'index' fails with mixed character sets
# New Ticket Created by Peter Gibbs # Please include the string: [perl #23752] # in the subject line of all future correspondence about this issue. # http://rt.perl.org/rt2/Ticket/Display.html?id=23752 > The 'index' opcode searches for a substring in another string. It seems reasonable to expect that equal strings would match at offset zero. But the program: set S0, "\xAB" find_chartype I0, "8859-1" set_chartype S0, I0 find_encoding I0, "singlebyte" set_encoding S0, I0 find_encoding I0, "utf8" find_chartype I1, "unicode" transcode S1, S0, I0, I1 eq S0, S1, equal print "not " equal: print "equal\n" index I0, S0, S1 print I0 print "\n" end yields the output: equal -1 which doesn't seem quite right. -- Peter Gibbs EmKel Systems
Re: [RfC] vtable->dump
On Sunday, September 7, 2003, at 06:21 , Nicholas Clark wrote: On Fri, Sep 05, 2003 at 09:34:04AM -0400, Dan Sugalski wrote: On Fri, 5 Sep 2003, Peter Haworth wrote: With the seen hash approach, I wouldn't expect the hash itself to take nearly as much space as the frozen structure; My experience tracing variable graphs in Perl 5 indicates otherwise. I think it may be worth listening to Dan on this one. Appended patch to Storable implements an option to count the size of the seen hashes. (There seem to be two, one for objects, one for classes) It's disabled by default - compile Storable with -DDEBUGME or change the #if 0 at line 23 of Storable.xs to enable. You'll also need Devel::Size installed. Taking a relatively complex structure, such as the POSIX exports: $ /usr/local/bin/perl5.8.0 -Mblib -MStorable -MPOSIX -MDevel::Size -le 'package POSIX; foreach ([EMAIL PROTECTED], [EMAIL PROTECTED], \%EXPORT_TAGS) { print Devel::Size::total_size $_; print length Storable::freeze $_; print $Storable::SEENSIZE; print ""}' reformatted, this give: size in memory size frozen seen hash size @EXPORT 22680 547719068 @EXPORT_OK 1943422 1864 %EXPORT_TAGS23686 596320472 Not very surprised by this result at all. Hash table entries are going to be roughly comparable in size to the original object graph, since so many objects in Perl are really quite tiny. The issues with the alternatives are stunningly painful, though. They've been hashed out already, though. I'm less concerned by transient memory consumption than by the sabotage of threading. Perhaps it would make sense to use a specialized data structure if memory consumption is really at the top of anybody's list of concerns. Remember the hash tables without linked lists chained off of them from undergrad CS? They're much more compact (literally a single table in memory), and most of their flukes are avoidable when entries can't be removed. — Gordon Henriksen [EMAIL PROTECTED]
Re: [RfC] vtable->dump
On Sunday, September 7, 2003, at 01:41 , Dan Sugalski wrote: At 8:56 PM -0600 9/4/03, Luke Palmer wrote: Gordon Henriksen writes: What you're suggesting also has significant side-effects: It halts hypothetical multithreaded programs, suspends DoD, prevents the traversal mechanism from calling back into parrot code, requires fine-grained locks which are extremely expensive and have been summarily abandoned to great acclaim in all prior works... and for that, still doesn't provide a useful form of thread safety in the general case anyhow. Is there a problem with providing a mechanism which would suspend all threads except for the "current" one, as to ensure that the serialize operates, er, serially. Could it be implemented with a top-priority event post that acquires a global lock? The biggest problem with a lock of this sort is that it requires all the interpreters to actually *check* it to be meaningful. (Not that there aren't other problems--there are, quite a number of them--but this is the big one) Locks nobody gets aren't useful, and for this one to be useful we'd need to have some assurance that the other threads were looking at it regularly, and that those threads were all paused when we wanted to do our stuff. You end up with either long delays waiting, or a lot of contention on a single global lock. Neither is particularly good. I think Luke wasn't suggesting a global lock, but rather suggesting that the competing threads be suspended using operating system facilities to temporarily prevent them from being scheduled. — Gordon Henriksen [EMAIL PROTECTED]
Re: Namespace'd subs in IMCC?
Joseph Ryan <[EMAIL PROTECTED]> wrote: > Leopold Toetsch wrote: >>It's not layed out what it should really do. I'm towards removing this >>directive and let the HLL compiler deal with it. > Well, in that case, would it be possible to allow ":" as a valid > character for use in symbol names? Yep. And probably some other chars used by HLL to mangle names. This should probably configurable somehow. > -Joe leo
Re: [RfC] vtable->dump
Steve Fink <[EMAIL PROTECTED]> wrote: > On Sep-08, Leopold Toetsch wrote: >> Parrot has some DOD counters for possibly active PMCs. A simple and >> probably fix-sized[1] hash based on such a count with just a mapping of >> {&pmc => id} takes 12 bytes per entry on 32-bit architecures and its >> really fast. > It's late so I'm probably being an idiot, but is all that is needed > (1) a unique id for every pmc address, and (2) whether or not the PMC > has been visited yet? If so, why not use the address itself for the id Yep. The address can serve as the ID (it would with Dan's scheme). > Or is there no simple way of going from a PMC address to its arena? With ARENA_DOD_FLAGS enabled: GET_ARENA(pmc). When its disabled it would need arena back pointers or a lookup like C. Both methods can generate an index usable for indexing into the bit vector. But I wouldn't want to use arena memory for the bit vector. This leads to the same multi threading issues, as Dan's scheme has. A bit vector instead of the hash allocated on the heap sounds very promising and space efficient. > Perhaps it's no win to pollute the cache with chunks of the bit vector > instead of the actual referenced PMC flagpoles. clone()/freeze()/dump() pull in the whole PMC. DOD doesn't for the fast case of just setting the live flag on a scalar. But ... > ... An > eighth of a bit per entry ought to be small enough, no? Yep. Think so. > I suppose I ought to pay more attention to the conversation (and the > current code) before butting in with suggestions. I've a feeling I'm > solving the wrong problem. No. The idea is fine and would work. leo
Re: Namespace'd subs in IMCC?
Brent Dax <[EMAIL PROTECTED]> wrote: > Leopold Toetsch: > # I'm not outerly sure, if imcc should even have such a functionality. > Or > # at least, if there is one, it should be configurable. The name > mangling > # is HLL dependent and there may be no general scheme to do it right for > # all kind of symbols. > Then allow a few more symbols in identifiers, and turn namespace into a > prefixing thing that applies to all identifiers: > .namespace Foo:: > .sub bar#really Foo::bar What about lexical names? Scopes? I think all IDs that should be mangled would need some kind of hint, that they are really subject of mangling. .namespace 1 "Foo::" .namespace 2 "_Foo::" .mangle sub 1 .mangle global 2 .sub bar # Foo::bar .global a# _Foo::a # how to generate @Foo::a and $Foo::a? If the HLL is integrated we could use callbacks to deal with that mess. leo
Re: Running .pbc files on Win32 and PIO_win32_open
Vladimir Lipskiy <[EMAIL PROTECTED]> wrote: > if (run_pbc == 2) { > +fclose(yyin); > pf = Parrot_readbc(interpreter, sourcefile); > if (!pf) > fatal(1, "main", "Packfile loading failed\n"); > Parrot_loadbc(interpreter, pf); > -fclose(yyin); Looks good. I'll commit that. (WRT other Win IO patches: Yes I don't want to interfer with Juergen's work on IO, 3-way rediffing is a PITA :) Thanks, leo
Dispatch on context
So, I was wondering about how to do multidispatch based on the context in which something is called, which leads me to wonder if there's going to be a way of reifying calling context so I could write: method whatever($arg1, $arg2) { my multi whatever ($self, Scalar $context: $arg1, $arg2) { ... } my multi whatever ($self, Array $context: $arg1, $arg2) { ... } whatever($_, want(), $arg1, $arg2) } Just wondering.
Re: Dispatch on context
Piers Cawley writes: > So, I was wondering about how to do multidispatch based on the > context in which something is called, which leads me to wonder if > there's going to be a way of reifying calling context so I could > write: > > method whatever($arg1, $arg2) > { my multi whatever ($self, Scalar $context: > $arg1, $arg2) { ... } > my multi whatever ($self, Array $context: > $arg1, $arg2) { ... } > whatever($_, want(), $arg1, $arg2) } The way I see that happening is if we allow certain kinds of constants in multi declarations. my multi whatever ($self, Class $context is derived(Scalar): $arg1, $arg2) {...} That'd be nice in a number of different ways. Without this special feature, an enabling level of indirection can be added with parameterized types: my class ClassVal [Class $TYPE] { # Does nothing, and may even be optimized away } my multi whatever ($self, ClassVal[Scalar] $context: $arg1, $arg2) {...} # ... whatever($_, ClassVal[want], $arg1, $arg2); # ^ does a : go here? That doesn't handle inheritance correctly, but it can be made to, provided proper introspection into $TYPE. Luke
Autovivification (was Re: E6: assume nothing)
Jonadab the Unsightly One said: > $s = %h{foo} = nonex; > > After deleting the foo key (and its value, if any) from %h this then > probably procedes to autovivify it when evaluating it as an rvalue; Now why on earth would you want to do that? Perl 5 doesn't. By the way, I trust this will be addressed (if it hasn't been already): perl5 -le 'print "gah!" if exists $a{b}{c}; print "phooey!" if exists $a{b}' perlfunc says: This surprising autovivification in what does not at first--or even second--glance appear to be an lvalue context may be fixed in a future release. -- Paul Johnson - [EMAIL PROTECTED] http://www.pjcj.net
Re: passing arguments to tests
--- Andrew Savige <[EMAIL PROTECTED]> wrote: > Fergal Daly wrote on 14 July 2003: > > is it possible with Test::Harness and MakeMaker to pass arguments to > > my test scripts? I think it's not but I just want to check for sure. > > The module I'm working on is getting a new "optimised" mode so I'd like > > to be able to run all the tests twice, once with and once without the > > optimiser. > > Which module is this? What did you end up doing? I'm interested to study > what you did, so as to learn new testing techniques. I do something like the following to get this effect: #!/usr/bin/perl -w use strict; use Test::Harness; use Getopt::Long; use Pod::Usage; GetOptions( 'help|?'=> sub { pod2usage(-verbose => 2); exit }, 'verbose!' => \$Test::Harness::verbose, 'quiet' => sub { $Test::Harness::verbose = 0 }, 'include=s' => \my @include, 'exclude=s' => \my @exclude, 'shuffle' => \my $shuffle_tests, 'fast' => \$ENV{FAST_TESTS}, ); In this case, this is part of my driver script and I call it like: grind --fast That runs through all of my tests, but the "FAST_TESTS" environment variable is available in the test programs that I run. Cheers, Ovid = Silence is Evilhttp://users.easystreet.com/ovid/philosophy/indexdecency.htm Ovid http://www.perlmonks.org/index.pl?node_id=17000 Web Programming with Perl http://users.easystreet.com/ovid/cgi_course/ __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com
Re: Parrot Z-machine
At 10:33 AM -0700 9/8/03, Amir Karger wrote: Before I start, a list question: is Google groups mailing list-aware, such that posting to Google's perl.perl6.internals group will email [EMAIL PROTECTED] Might be more convenient for me than reading stuff on Google & then logging in to my Yahoo account to post. The NNTP gateway's pretty much one way. You might try using the news server at nntp.perl.org, though there may still be a delay. News, unfortunately, is rather badly prone to abuse and spamming. --- Dan Sugalski <[EMAIL PROTECTED]> wrote: On Sat, 6 Sep 2003, Amir Karger wrote: > I'll need to write Zmachine.ops, or some such. It will include all > the Z-machine operations, which the bytecode will call. Yep. OK. Although Luke Palmer seems to think differently. That's OK--there are a number of different ways to go about this. He said: I think a z-machine to parrot converter, making some of the more complex ops sub calls or something, would be best. We need to work with the z-machine bytecode directly, though, because many games are distributed without source. That sounds more like my "disassemble Zcode to Z assembly, then translate that to PASM+, i.e., PASM that calls Zcode ops in addition to the regular ops, but uses the Parrot registers et al., then compile and run that." Which you can do, and ultimately is probably the best thing to do. (It is, actually, what I'm suggesting you do, though I think I'm probably skipping a few bits) Though the possibility of writing a z-machine interpreter in parrot assembly, IMCC, or one of the languages that targets parrot is definitely an option. Of course, between these two solutions, there would be some overlap in the Z-specific opcodes, but there's the major question of whether we go through PASM or not. But as Luke (and my-ticket-to-fame Piers Cawley) mention, not using PASM loses you some fancy optimizations. (At least I think that's what they said.) Are these optimizations unnecessary if we write the whole bytecode implementer in C? Nah, I think it's a matter of some interpretation. Piers was assuming that if you used a *loadable* opcode library that you couldn't be JITted. That's not the case, though I'm not sure it's a huge issue anyway--there's a limit to how fast Lurking Horror can run, after all. :) I'm happy to do (or, maybe more likely, try and fail miserably to do) either method, but The List should probably figure out a direction for me to go in first. (Given my knowledge of Parrot, I'm probably the least qualified person on this list right now to make this decision. Unless you tell me to do whichever I think will be more fun, in which case I'll write a Z-code to BASIC translator) Here's what *I* think would be cool. 1) We have a set of z-machine ops to implement whatever bits you need that are otherwise inconvenient to implement in what ops we have now. (There may not be any, which is fine) 2) We have whatever support parrot assembly routines you might need. (Library code for the interpreter and whatnot) 3) We have code, written either in C or parrot assembly, that can turn a z-machine file into parrot bytecode, parrot assembly, or IMCC source. (This code should rely only on ANSI C89, the C89 libraries, and/or the parrot library routines, and shouldn't do any I/O) 4) We have code that can definitively identify a file as z-code. (This may be the fact that there's a -b:zcode switch immediately before it) What we do then is weld this all together, and it then gives us the ability to dynamically load up and run z-code files. The flow will be that parrot opens the bytecode file and tries to identify it. Using the code in #4, it identifies it as z-code, and then hands it to the code for #3 (which is a parrot bytecode loader). #3 translates it to parrot bytecode (directly or indirectly) including the support code from #2 (if there is some) and hands it to the interpreter to execute. The interpreter loads up the opcode library from #1 if it needs to, and goes. > > > we want parrot to be able to directly load Z-code files, rather > > than having to first run an external program to convert them. > > Right. The problem that I see with this is that Z-code "story" > files have a very definite header format, which is almost but not > quite entirely unlike Parrot bytecode. Just for example, the first > few bytes are totally different, but are necessary for both > languages. Right, which is good. What I want to have happen is that when parrot is handed a bytecode file to execute, it examines the header of the file to find out what type of bytecode it is, and dispatch to the right loader. So when you load up a story file as if it were native bytecode, the bytecode loading routines identify it as a zcode story file rather than parrot bytecode and dispatch it to the zcode loader rather than the parrot bytecode loader. Er, I'll assume you have a magic (pun slightly intended) way to decide which files are
Re: Of AST and YAL
--- Leopold Toetsch <[EMAIL PROTECTED]> wrote: > James Michael DuPont wrote: > > > > The redland rdf api provides a good C api into rdf. > > > > The advantages of using rdf instead of a homegrown format are the > > amount of tools available, you then have filters, and visualization > > tools immediatly available. > > First I have to admit: I don't like XML. I dont either. ;) XML sucks. Logic rulez. I have been using timbls n3 notation http://www.w3.org/2000/10/swap/Primer >Second I don't want an > external > library at such a central place inside Parrot. Yes. We can make a native structure. The only thing is this basic api. We dont need to 1. you represent the ast as a set of triples (maybe quads with context) so each thing is a statement like : subject predicate object [context]. 2. You can transform or flatten any ast into a set of triples that is the only real support that I need from you. Basically a set of functions that can be used to emit any ast as a set of logical statements. The logical statements can just be parrot arrays and the user can figure out what to do with them. 3. you can build an ast out of such triples. The only support needed is to have some way to lookup the type of predicate by a name, or to reference an ast node by id. Then we can write various parser in parrot that use this api. I guess the input could also be a parrot array with three columns. 4. You register new predicate names (i have been extracting them from the gcc) you can see some of those here http://introspector.sourceforge.net/2003/09/ildasm_header_micront.owl That could also be an array of known predicates. Each statement could just contain the index to them. 5. we dont need an rdf parser or n3 parser, but can factor that out like redland does : in fact, the only api needed to be implemented is the rdf.storage and rdf.model. those two represent the parsed state of an rdf document and the way to interface to the representation. The rest can be left out of the core. >I also think (hvaing a > > short look at redland-0.9.14, that this might be some overkill to use > > that as the AST interface for parrot. yes. We can make a very cut down api and simple (parrot) provider for n3. The Eulersharp code can be compiled later to parrot via dotgnu. http://eulersharp.sourceforge.net/2003/03swap/ > > OTOH ... > > > > Later on when we have a two-way gateway, then you will be able to > use a > > proof engines to do rule-based tweaking of your AST before sending > this > > off to the backend for execution. > > ... ab interface to RDF would be for sure a nice to have. Yes, I am really interested in the ability to plug in a proof engine and feed it rules to tweak the code. The format is really *egal*. (sorry for the extreme-denglishing) > > > > I hope that you do not get annoyed at this suggestion, if you feel > it > > is off topic, please tell me. > > No, not at all. Thanks for your input. Ok, then you get more. :) schöne Grüsse aus Frankfurt am Main, mike = James Michael DuPont http://introspector.sourceforge.net/ __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com
FEATURE FREEZE Sunday 14-Sep-2003
Had to stick that year on in case this message gets unearthed. Ok, the little voices in my head have told me that it's time for that release we were talking about. So, you have until Sunday to cram in any functionality you want in the next release. After Sunday, it will be cleanups, bugfixes, and completing existing functionality only for a little while. The version number has not yet been decided -- 0.0.11, 0.1.0, or maybe 4.3 for the hell of it. The code name is, of course, open for discussion. Although I reserve the right to call it the "hairy tulip" release if I want. (in private) (with the shades drawn)
Missing Ops
While playing with YAL (Yet Another Language) I'm discovering a lot of asymmetries in ops files: neg_p is missing (neg_p_p only, but neg_i is there) mod_p_p_i, mod_p_p_n are missing (add, sub, ... have these variants) keyed assign ops are missing (set only) The question is what to do: 1) should these ops just be implemented or 2) should the assembler work around missing ops 3) should each HLL work around missing ops leo
This week's summary
The Perl 6 Summary for the week ending 20030907 Welcome to the last Perl 6 summary of my 35th year. Next week's summary will (in theory) be written on my 36th birthday (a year of being square, so no change there then). I'll give you fair warning that it might be late, though it probably won't. Newcastle University has, in its infinite wisdom decided to have its students enrolling on that day so Gill will be off up to Newcastle to register leaving me at home with nothing to do but keep the house tidy in case a buyer wants to come and look at it, so sitting in one place with a laptop writing a summary seems like a good strategy. As last week's 'world turned upside down' approach of starting with perl6-language was such a success we'll do the same again this week. The language list gets some traffic shock! Jonadab the Unsightly One replied to Abhijit A. Mahabal's message from the first of August concerning junctions and set theory. http://xrl.us/sm6 Meanwhile, on perl6-internals Serialization is Hard! Last week's discussion of serialization sparked off by Leopold Tötsch's suggestion of a "vtable->dump" mode *really* got into its stride this week. It turns out that getting this right is a Hard Problem in the presence of threads. Dan's plan for serialization involves using the GC's object graph walker to work out what to serialize when you tell Parrot to dump a PMC. Leo worried that this would essentially stop the garbage collector running during serialization which could be tricky if the serialization process tried to allocate any memory. Dan and Leo ended up in a protracted, but polite, argument about details. At about 45 entries into the thread, Leo produced a summary of the various options and issues associated with them. http://xrl.us/sm7 http://xrl.us/sm8 -- Leo's summary File Spec Leo Tötsch commented on Vladimir Lipskiy's implementation of a File::Spec like tool for Parrot. (File::Spec is Perl's tool for dealing with filenames and paths in a platform independent fashion). Michael Schwern pointed at Ken Williams' "excellent Path::Class module which gives you actual file and directory objects" which he reckons has a much better interface than File::Spec. http://xrl.us/sm9 Notifications Gordon Henriksen posted a great discussion of using notifications to implement weakrefs. Rather wonderfully he used the notification system itself as a good example of why dying object notifications were a good idea. http://xrl.us/sna Parrot 100% GNU .NET Danger. Here be Licensing Issues. I don't do Licensing issues. The main thrust of the discussion was what kind of library would ship with Parrot. Dan's answer is worth reading, if only for the "That's a swamp I don't have enough castles for" line. http://xrl.us/snb http://xrl.us/snc -- Dan's take on the library You are in a maze of keyed variants, all similar This seems to have been a week in which Dan and Leo spent a good deal of their time politely disagreeing with each other. This time they were disagreeing about the need for all the keyed variants of Parrot's opcodes. Dan outlined the reasoning behind demanding keyed variants of every operation in a PMC's vtable (Executive summary: A combination of speed and space reasons). Leo still doesn't seem convinced but, for now, Pumpking trumps Patch monster. http://xrl.us/snd Parrot Z-machine Amir Karger's post from last week about implementing the Z-machine (the VM that runs Infocom and other text adventures) got de-Warnocked this week. Nicholas Clark explained that doing the Z-machine 'properly' would require some bits of Parrot that weren't actually there yet, specifically dynamic opcode loading and dynamic bytecode conversion. This led to a discussion of how to get those things implemented. http://xrl.us/rtr PIO Questions Benjamin Goldberg posted a long list of issues and suggestions about handling character type and encoding on Parrot IO objects. Jürgen Bömels said that there were indeed issues, that he'd be dealing with them as tuits allowed and that patches are welcome. http://xrl.us/sne How to dynamically add a method to a class Joseph Ryan had asked how to add a method to a class at runtime. Dan explained what was supposed to happen (each class has a 'backing namespace' associated with it which contained all the class's methods). Leo asked for a few details about how that would look in Parrot assembly. A little later, Joseph reported what appeared to be a bug in the way IMCC handles ".namespace". It appears that IMCC is working as designed, the question is whether the design is doing the Right Thing. http://xrl.us/snf http://xrl.us/sng Proposed amendment to charty
Re: Missing Ops
Leopold Toetsch sent the following bits through the ether: > 1) should these ops just be implemented or At this point we want to make it really easy to target parrot. I know it's not hard to work around these exceptions, but I reckon implementing the complete set of ops for now would be a good idea. We can always move the logic into the assembler later if we decide to prune ops. ObLeonBrocard: he just be restin', arrr -- Leon Brocard.http://www.astray.com/ scribot.http://www.scribot.com/ ... I was young, I needed the money
Re: Of AST and YAL
James Michael DuPont wrote: The redland rdf api provides a good C api into rdf. The advantages of using rdf instead of a homegrown format are the amount of tools available, you then have filters, and visualization tools immediatly available. First I have to admit: I don't like XML. Second I don't want an external library at such a central place inside Parrot. I also think (hvaing a short look at redland-0.9.14, that this might be some overkill to use that as the AST interface for parrot. OTOH ... Later on when we have a two-way gateway, then you will be able to use a proof engines to do rule-based tweaking of your AST before sending this off to the backend for execution. ... ab interface to RDF would be for sure a nice to have. I hope that you do not get annoyed at this suggestion, if you feel it is off topic, please tell me. No, not at all. Thanks for your input. Thanks, mike leo
Re: Missing Ops
On Mon, 8 Sep 2003, Leopold Toetsch wrote: > While playing with YAL (Yet Another Language) I'm discovering a lot of > asymmetries in ops files: > > neg_p is missing (neg_p_p only, but neg_i is there) > mod_p_p_i, mod_p_p_n are missing (add, sub, ... have these variants) > keyed assign ops are missing (set only) > > The question is what to do: > 1) should these ops just be implemented or This one. We can consider pruning out unused variants later, after we've been up and running and have reasonable HLL output to check against. Dan
Re: Parrot Z-machine
Before I start, a list question: is Google groups mailing list-aware, such that posting to Google's perl.perl6.internals group will email [EMAIL PROTECTED] Might be more convenient for me than reading stuff on Google & then logging in to my Yahoo account to post. --- Dan Sugalski <[EMAIL PROTECTED]> wrote: > On Sat, 6 Sep 2003, Amir Karger wrote: > > > I'll need to write Zmachine.ops, or some such. It will include all > > the Z-machine operations, which the bytecode will call. > > Yep. OK. Although Luke Palmer seems to think differently. He said: > I think a z-machine to parrot converter, making some of the > more complex ops sub calls or something, would be best. We need to work > with the z-machine bytecode directly, though, because many games are > distributed without source. That sounds more like my "disassemble Zcode to Z assembly, then translate that to PASM+, i.e., PASM that calls Zcode ops in addition to the regular ops, but uses the Parrot registers et al., then compile and run that." Of course, between these two solutions, there would be some overlap in the Z-specific opcodes, but there's the major question of whether we go through PASM or not. But as Luke (and my-ticket-to-fame Piers Cawley) mention, not using PASM loses you some fancy optimizations. (At least I think that's what they said.) Are these optimizations unnecessary if we write the whole bytecode implementer in C? I'm happy to do (or, maybe more likely, try and fail miserably to do) either method, but The List should probably figure out a direction for me to go in first. (Given my knowledge of Parrot, I'm probably the least qualified person on this list right now to make this decision. Unless you tell me to do whichever I think will be more fun, in which case I'll write a Z-code to BASIC translator) > > > we want parrot to be able to directly load Z-code files, rather > > > than having to first run an external program to convert them. > > > > Right. The problem that I see with this is that Z-code "story" > > files have a very definite header format, which is almost but not > > quite entirely unlike Parrot bytecode. Just for example, the first > > few bytes are totally different, but are necessary for both > > languages. > > Right, which is good. > > What I want to have happen is that when parrot is handed a bytecode > file to execute, it examines the header of the file to find out what > type of bytecode it is, and dispatch to the right loader. So when > you load up a story file as if it were native bytecode, the bytecode > loading routines identify it as a zcode story file rather than > parrot bytecode and dispatch it to the zcode loader rather than the > parrot bytecode loader. Er, I'll assume you have a magic (pun slightly intended) way to decide which files are Zcode? I mean, sure, if the rule is "anything that doesn't match a Parrot header", you're fine, but once you've included Python bytecode, Java bytecode, and compiled Befunge bytecode (a man can dream), how will you tell the Zcode from other bytecode noise? I don't see anything particularly obvious in the header contents: http://www.inform-fiction.org/zmachine/standards/z1point0/sect11.html I assume having a separate Z-code loader solves the "Z-code has two-byte words" problem? OTOH, that means even more of Parrot functionality we don't get to use. > > From what I can understand, "native" means running directly from the > > bytecode, but writing new ops. That is, not using PASM at all. In > > that case, do I even have access to Parrot's stack? (Or do I just > > need to access all of it through C?) > > PASM is parrot's assembly, but when you're writing actual op functions > you're extending parrot's opcodes--C is essentially Parrot's > microcode. From within opcode functions you have full access to all > of the underlying structures, so if you want to access Parrot's stack > in z-machine ops, go for it. Hm. Well, *do* I want access Parrot's stack? Something to think about if I ever start actually writing code for this project, instead of talking about it. -Amir __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com
Re: Moving imcc ...
On Mon, Sep 08, 2003 at 02:24:41PM +0200, Leopold Toetsch wrote: > The normal way to move a file is to copy OLD to NEW, and then issue > the normal CVS commands to remove OLD from the repository, and add NEW > to it. > > $ mv OLD NEW > $ cvs remove OLD > $ cvs add NEW > $ cvs commit -m "Renamed OLD to NEW" OLD NEW > or better - including the comments WRT "commit -r rev": > $ cvs commit -m "Renamed OLD to NEW" -r $next_rev OLD NEW > > Is there a better way to move the files? The disadvantage with the method that you describe is that the revision history of the files is lost. An alternative is that someone with shell repository access on cvs.perl.org moves the files around in the repository, which has the advantage of preserving the revision history in the new files, but the disadvantage of breaking the repository for anyone attempting to pull out historical snapshots. I'm not sure if the suggested solution to this is to copy the files inside the repository (rather than moving them) and then use cvs -f remove OLD to delete the files in their old location, which would mean CVS is still able to create valid historical snapshots. (valid in as much as they will compile - they won't be perfect because extra files will appear in the new locations, as if they had been committed there since the beginning of time, rather than appearing that at the time of the move) CVS sucks for this sort of stuff. Nicholas Clark
Re: IMCC register assigment bug?
On Sep-08, Leopold Toetsch wrote: > Steve Fink <[EMAIL PROTECTED]> wrote: > > I'm getting my register stomped on in the following code: > > > .sub _bogus > > set $P1, $P0[0] > > find_lex $P2, "x" > > I've fixed that part now. $P1 didn't properly interfer with $P2, so both > got the same register. Thank you! It was dumb code generation in the first place, and I have now fixed it so that $P2 *is* used later on, but I suspect I would have run into it again eventually.
Re: Moving imcc ...
Dan Sugalski wrote: On Mon, 8 Sep 2003, Leopold Toetsch wrote: Imcc is still lacking full integration inside Parrot. To accomplish this, we would need these steps: - rebuild and commit directory structure $Parrot_Root/imcc - move files from languages/imcc to ../imcc [1] - adapt Makefiles - include imcc source files except main.c into libparrot Almost. IMCC's getting integrated with Parrot. All the core sources should be in one spot, so either move the current top-level C code to core/ and put the IMCC stuff in there as well (in core/ directly, not core/imcc/) or move all the IMCC files to the top level directory (with the header files in include/parrot) Could be done too (with some file renaming i.e. jit.c) but imcc has its own test structure and its own TestCompiler.pm. OTOH pd07 has something about subsystem - they should be separate. As imcc is the assembler/compiler subsystem I'd rather have it in a separate directory. ... The makefile needs to be integrated into the top-level makefile. (Again, in the top-level. I don't want a separate makefile for IMCC) Nor do I. There's not much reason to have IMCC split out, so I'd rather not. pdd07, file name clashes, t/* Dan leo
Re: Of AST and YAL
--- Leopold Toetsch <[EMAIL PROTECTED]> wrote: > Michal Wallace <[EMAIL PROTECTED]> wrote: > > (My plan for this week was to do something very similar, > > and try to get a simple lisplike language to integrate > > with python) > > I'm thinking of a AST text interface for imcc/parrot too. The lispish > AST-dump produced by YAL is handy for debugging and testing, and it > can > be parsed easily to reconstruct the AST again. OK. This is where I would like to get involved. Currently I have been working on documenting the GCC ast, in hopes of making that generatly useful. See my last mail about that here : http://sourceforge.net/mailarchive/forum.php?thread_id=3080551&forum_id=7974 My plan is to make a generic RDF interface to save, manip. and restore ASTS. This will also include some of the parrot languages. The redland rdf api provides a good C api into rdf. The advantages of using rdf instead of a homegrown format are the amount of tools available, you then have filters, and visualization tools immediatly available. Later on when we have a two-way gateway, then you will be able to use a proof engines to do rule-based tweaking of your AST before sending this off to the backend for execution. The other advantage of using redland is that you can exchange ASTs in memory or using one of the storage systems that plug-in like the berkleydb interface. That allows high-speed exchange of large amounts of ASTS. I hope that you do not get annoyed at this suggestion, if you feel it is off topic, please tell me. Thanks, mike = James Michael DuPont http://introspector.sourceforge.net/ __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com
Re: [perl #23751] [PATCH] f should be if in genclass.pl
Michael Scott <[EMAIL PROTECTED]> wrote: > f should be if on line 41 of parrot/classes/genclass.pl. Thanks, applied. leo
Re: Parrot Z-machine
Luke Palmer <[EMAIL PROTECTED]> writes: > Zellyn Hunter writes: >> So I take it the goal is to to teach parrot to understand z-machine >> opcodes, rather than simply writing a z-machine interpreter that >> runs on parrot, or rewriting inform to compile to parrot? > > I doubt it. I think a z-machine to parrot converter, making some of the > more complex ops sub calls or something, would be best. We need to work > with the z-machine bytecode directly, though, because many games are > distributed without source. > > And perhaps, once dynamic opcode loading is ready, those sub calls can > turn into real ops for speed. But that's not important at the moment: > the important thing (well, as important as z-machine can be :-) is to > get the translator working. Turning them into real ops might not be the right thing to do in the presence of JIT of course.
Re: IMCC register assigment bug?
Steve Fink <[EMAIL PROTECTED]> wrote: > I'm getting my register stomped on in the following code: > .sub _bogus > set $P1, $P0[0] > find_lex $P2, "x" I've fixed that part now. $P1 didn't properly interfer with $P2, so both got the same register. > Steve leo
Re: Parrot Z-machine
On Mon, 8 Sep 2003, Piers Cawley wrote: > Luke Palmer <[EMAIL PROTECTED]> writes: > > Zellyn Hunter writes: > >> So I take it the goal is to to teach parrot to understand z-machine > >> opcodes, rather than simply writing a z-machine interpreter that > >> runs on parrot, or rewriting inform to compile to parrot? > > > > I doubt it. I think a z-machine to parrot converter, making some of the > > more complex ops sub calls or something, would be best. We need to work > > with the z-machine bytecode directly, though, because many games are > > distributed without source. > > > > And perhaps, once dynamic opcode loading is ready, those sub calls can > > turn into real ops for speed. But that's not important at the moment: > > the important thing (well, as important as z-machine can be :-) is to > > get the translator working. > > Turning them into real ops might not be the right thing to do in the > presence of JIT of course. Nah, that's not an issue. The JIT has to be able to handle loadable op libraries. It can TIL the unknown ops easily enough, and we can make sure sufficient information is attached to the op libraries to allow real JITting as well. Dan
Moving imcc ...
Imcc is still lacking full integration inside Parrot. To accomplish this, we would need these steps: - rebuild and commit directory structure $Parrot_Root/imcc - move files from languages/imcc to ../imcc [1] - adapt Makefiles - include imcc source files except main.c into libparrot [1] info cvs * Adding and removing:: Adding/removing/renaming files... * Moving files::Moving and renaming files * Outside:: The normal way to Rename The Normal way to Rename The normal way to move a file is to copy OLD to NEW, and then issue the normal CVS commands to remove OLD from the repository, and add NEW to it. $ mv OLD NEW $ cvs remove OLD $ cvs add NEW $ cvs commit -m "Renamed OLD to NEW" OLD NEW or better - including the comments WRT "commit -r rev": $ cvs commit -m "Renamed OLD to NEW" -r $next_rev OLD NEW Is there a better way to move the files? Does someone have a script for that kind of stuff? Thanks, leo
Re: Parrot Z-machine
On Sat, 6 Sep 2003, Amir Karger wrote: > OK. I think I've learned enough about Parrot to respond to this email > intelligently. > > --- Nicholas Clark <[EMAIL PROTECTED]> wrote: > > > - Is it not being ported because of a lack of tuits, or because > > > it's extremely hard? > > > > We'd need dynamic opcode loading because we don't want to have the > > Z-machine specific opcodes compiled into every parrot (or every other > > specialist set of opcodes) > > Right. What you're saying here is that I'll need to write Zmachine.ops, > or some such. It will include all the Z-machine operations, which the > bytecode will call. Yep. > > We'd want dynamic bytecode conversion because we want parrot to be > > able to directly load Z-code files, rather than having to first run > > an external program to convert them. > > Right. The problem that I see with this is that Z-code "story" files > have a very definite header format, which is almost but not quite > entirely unlike Parrot bytecode. Just for example, the first few bytes > are totally different, but are necessary for both languages. Right, which is good. What I want to have happen is that when parrot is handed a bytecode file to execute, it examines the header of the file to find out what type of bytecode it is, and dispatch to the right loader. So when you load up a story file as if it were native bytecode, the bytecode loading routines identify it as a zcode story file rather than parrot bytecode and dispatch it to the zcode loader rather than the parrot bytecode loader. > From what I can understand, "native" means (if we ignore dynamic > bytecode > conversion issues) running directly from the bytecode, but writing new > ops. That is, not using PASM at all. In that case, do I even have > access > to Parrot's stack? (Or do I just need to access all of it through C?) I > guess maybe my mistake is in thinking of PASM as Parrot. PASM is parrot's assembly, but when you're writing actual op functions you're extending parrot's opcodes--C is essentially Parrot's microcode. >From within opcode functions you have full access to all of the underlying structures, so if you want to access Parrot's stack in z-machine ops, go for it. Dan
Re: [RfC] vtable->dump
On Monday, September 8, 2003, at 03:34 , Steve Fink wrote: On Sep-08, Leopold Toetsch wrote: I don't know why Storable.xs is using a general HV for mapping an address to an integer (more isn't needed to serialize all IMHO - Dan's next_for_GC approach doesn't provide more info). Perl5 does convert the address to a string and uses all weirdness of hv.c code. This isn't necessary for this special kind of problem. Parrot has some DOD counters for possibly active PMCs. A simple and probably fix-sized[1] hash based on such a count with just a mapping of {&pmc => id} takes 12 bytes per entry on 32-bit architecures and its really fast. It's late so I'm probably being an idiot, but is all that is needed (1) a unique id for every pmc address, and (2) whether or not the PMC has been visited yet? If so, why not use the address itself for the id (or some variant, like index of the pmc in its arena, added to/or-ed with some base id for the arena itself)? A down side to using the address as a key is that it would cause even the simplest serialization to have nondeterministic (or at least unpredictable) output, which certainly makes testing more difficult. And for the flag, have arenas preallocate a bit vector of seen flags for all their PMCs? (Or if you have the arena base ids, then you can have a single bit vector for all arenas.) Consider concurrent serialization from multiple threads. The seen hash (table, whatever) is local to the serialization, and thus thread-safe. The programmer can guarantee that the traversed structures are not concurrently updated—the runtime flatly cannot. Any global structure is inherently not thread-safe. Furthermore, global structures are persistently extant, and must be paid for continually. Most programs will never serialize even one object graph, much less spend any significant portion of their CPU time in serialization. — Gordon Henriksen [EMAIL PROTECTED]
Re: Moving imcc ...
On Mon, 8 Sep 2003, Leopold Toetsch wrote: > Imcc is still lacking full integration inside Parrot. To accomplish > this, we would need these steps: > > - rebuild and commit directory structure $Parrot_Root/imcc > - move files from languages/imcc to ../imcc [1] > - adapt Makefiles > - include imcc source files except main.c into libparrot Almost. IMCC's getting integrated with Parrot. All the core sources should be in one spot, so either move the current top-level C code to core/ and put the IMCC stuff in there as well (in core/ directly, not core/imcc/) or move all the IMCC files to the top level directory (with the header files in include/parrot) The makefile needs to be integrated into the top-level makefile. (Again, in the top-level. I don't want a separate makefile for IMCC) There's not much reason to have IMCC split out, so I'd rather not. Dan
Re: Parrot Z-machine
> "AK" == Amir Karger <[EMAIL PROTECTED]> writes: AK> Er, I'll assume you have a magic (pun slightly intended) way to AK> decide which files are Zcode? I mean, sure, if the rule is AK> "anything that doesn't match a Parrot header", you're fine, but AK> once you've included Python bytecode, Java bytecode, and compiled AK> Befunge bytecode (a man can dream), how will you tell the Zcode AK> from other bytecode noise? I don't see anything particularly AK> obvious in the header contents: AK> http://www.inform-fiction.org/zmachine/standards/z1point0/sect11.html i see plenty there than can be used to identify a zcode file. it would take a little work to select a set of interesting fixed offset bytes and their normal range of values. then writing the magic detector is easy. the unix program file does it that way right now and it is fairly decent. /etc/magic has a large table of bytes and program types. also we can have a short script read a normal zcode file and put it in a special and marked section (using the standard directory stuff) of a parrotcode file from which it can be easily loaded and run. in fact this is a general tool that can wrap any other format code into something parrot could load and then invoke the appropriate byte code converter. AK> I assume having a separate Z-code loader solves the "Z-code has AK> two-byte AK> words" problem? OTOH, that means even more of Parrot functionality we AK> don't get to use. it all depends on how the zcode interpreter is built. as others have said there are many choices and many overlapping ideas. so i doubt we would all agree on the best way to do this. the designs range from a total code conversion, load and translate the zcode into equivilent imcc. this should be the easiest to do as you just need to write a code generator for each zcode op. you can fake a stack in imcc using a PMC array or someother technique. just have all the zcode stack opcodes use the pmc based stack in the translation. this could be done in pure perl as well and run offline to generate imcc code. this would still run directly on parrot but use its existing set of opcodes note that parrot can't directly run zcode because of the 2 byte issue. a real zcode loader would have to read in the zcode (from a file or maybe that marked section i mentioned before) and translate it to legal parrot opcodes in 32 (or 64) bit sizes. then you would need to write real code to handle each of the needed parrot opcodes. i would consider this much more work but it can be truly said that parrot is running zcode natively. and of course you can have a mix of the above two. the main system will translate simple zcode ops to equivilent short pieces of imcc code. then the heavier duty zcode ops can be written in c and loaded on demand. these new codes can be generated by the translator when it sees the zcodes that need them. by following that path, you can start with a pure translator (first design) and then migrate over to a native interpreter (second design) but as incrementally as desired. AK> Hm. Well, *do* I want access Parrot's stack? Something to think AK> about if I ever start actually writing code for this project, AK> instead of talking about it. well you need a stack somewhere. you can acces parrot internal stacks from any parrot ops that you create (in c). regular parrot ops only some limited access to those stacks. but you can use your own stacks in the registers and with arrays which would mean both parrot ops and your own ops would be able to access them. hope this clarified the issues, uri -- Uri Guttman -- [EMAIL PROTECTED] http://www.stemsystems.com --Perl Consulting, Stem Development, Systems Architecture, Design and Coding- Search or Offer Perl Jobs http://jobs.perl.org Damian Conway Class in Boston - Sept 2003 -- http://www.stemsystems.com/class