Generic callback mechanism in NCI
I started to write an OpenGL library and was only a couple of dozen lines into the pir when I remembered the documentation about callbacks in docs/pdds/draft/pdd16_native_call.pod . Currently there are only two signatures supported on callbacks: one with user_data and extern_data and the other with extern_data and user_data (the positions are all that is different). These are nice and general functions and work well because the user_data is/should be opaque to the calling C function, the user_data provides a place to store the PMC, and the user_data is used to lookup the interpreter that should be used to run the sub stored in the PMC. The pdd says that outside the two provided signatures, anybody wanting to implement NCI connections to callback functions will need to do some hand C coding. Hand C coding isn't at all bad, but it would be nice to have a generic mechanism for storing callbacks from C functions that don't provide a slot for opaque user_data. I was on my way to coding a solution with my meager C skills and wanted to make sure that what I was doing was sane (for some definition of the word sane). So my proposal goes something like this: - Get a new unique id from a new op called get_callback_id - Call a variant of the new_callback op, passing in the unique id - The new_callback variant stores the user data under the unique id - The callback references a unique C function that will return that unique id - When the callback is fired, the unique id is used to lookup the user_data - Any external data is parsed as needed - The registered sub is played Ok, sounds hard enough - but it gets worse. Here is how I would implement this generic callback system and what I think would be required. There will probably need to a file similar to src/call_list.txt such as src/callback_list.txt with entries similar to: 10 v v 10 v ii Where the first number is the number of functions to pre-create, and the other two parts are the return type and signature. I'd rather not have the number be hardcoded and go with a jit solution or some other solution for runtime C function generation - but I'm not aware of anything else that would work besides pre-compiled C functions. I think it would be nice if libraries could generate/compile the needed functions independent of anything hardcoded (and that goes for call_list.txt too for that matter). The entires in callback_list.txt would generate functions similar to the following in nci.c # parrot callback functions void pcbf_v_JV_0(void) # depending up signature void pcbf_v_JV_1(void) ... void pcbf_v_JV_9(void) I would then add two more ops (I shudder to say that - I'm not sure if adding ops is a frowned upon thing). Those ops are (I haven't played with it to know if this is the right format): op get_callback_id(out INT, in STR) # the output is a unique INT for the given signature in STR # it would fail if called more times than the 10 listed # in callback_list.txt (unless a jit solution could be implemented) op delete_callback_id(in INT, in STR) # deletes the user_data in the storage structure, freeing it # for later use Currently the following op is defined in src/ops/core.ops: op new_callback(out PMC, invar PMC, invar PMC, in STR) I want to add one more variant of this op op new_callback(out PMC, in INT, invar PMC, invar PNC, in STR) Another piece that is required is that there be a global ResizablePMCArray (actually there may be multiple based on the registered signatures). Again, I can hear everybody shuddering at the mention of a global. I don't know enought about parrot yet to know if it should be a true global variable, or if it should be tied to an interpreter, or if there is somewhere I should register the PMC with, or if there is already a central structure that would take care of functions like this. My questions are: - Does anybody else want a generic callback function mechanism in NCI? - Is this a relatively sane/clean way to do it? - Is there a better way to generate the functions for each signature? - What is the right way to store that global user_data until the callbacks are fired? I don't think I've been clear enough, but I'll post and then answer questions. I think that it would be good to have something that libraries could use without having to drop down to the C level - I just am not sure if this is too much bloat to implement it. Paul Seamons I've been here all along - I'm just really quiet.
Re: [perl #46541] [BUG] t/pmc/objects.t test failures
On Friday 19 October 2007 09:30:31 Bernhard Schmalhofer wrote: > Why should it be a TODO test? For one, there's no ticket for it. For two, allowing known failing bugs is the Broken Windows anti-pattern. Not only does it undermine confidence in the software (oh, still broken!) but it adds noise that makes it far more difficult to know if a change introduced a new bug (which tests were failing? 1, 3, and... oh forget it!) > As far as I see, it exposes a real bug. So I want a red light going on. That's why we have the bug tracker. > TODOing known bugs for a release is a different issue. I don't see why. I'd like to see Parrot be in a releasable state with every checkin. That's why we have the tests. -- c
[svn:parrot-pdd] r22276 - trunk/docs/pdds/draft
Author: kjs Date: Fri Oct 19 12:40:49 2007 New Revision: 22276 Modified: trunk/docs/pdds/draft/pdd06_pasm.pod Log: and yet another line too long in pdd06_pasm.pod Modified: trunk/docs/pdds/draft/pdd06_pasm.pod == --- trunk/docs/pdds/draft/pdd06_pasm.pod(original) +++ trunk/docs/pdds/draft/pdd06_pasm.podFri Oct 19 12:40:49 2007 @@ -49,7 +49,8 @@ allison: which is what we want, anyway, right? particle: yup yes - good, looks like we're in agreement and headed in the proper direction on that topic. + good, looks like we're in agreement and headed in the proper direction on +that topic. =back
[perl #46503] [TODO] Remove individual runcore command line flags...
i've added -R, --runcore options to parrot in r22284, with documentation and tests. see C and C for details. next we need to decide what to do about the old options. when do we deprecate them? how do we migrate to the new syntax? etc. if you have ideas, let them be known by replying here. i suggest we migrate all existing code to the new options at once, and mark the old options as deprecated. however, i'd like to see the old options stay around for a while, perhaps with a runtime warning after the 0.5.0 release, a fatal error after 0.6.0, and removal in 0.7.0. those last two release numbers are flexible. your thoughts? ~jerry
Re: [perl #46481] [PATCH] Add a GC-Debugging Core
On 10/16/07, via RT chromatic <[EMAIL PROTECTED]> wrote: > # New Ticket Created by chromatic > # Please include the string: [perl #46481] > # in the subject line of all future correspondence about this issue. > # http://rt.perl.org/rt3/Ticket/Display.html?id=46481 > > > > Here's a nifty little patch that adds a new runcore which runs a full GC cycle > after each op. As you might expect, it takes a bit longer than normal tests > (so I don't really recommend it for running the shootouts), but it has the > possibility of finding GC bugs very thoroughly. ... > nice! i've modified the runcore option for parrot, so this patch won't apply as is. it's no big deal to change it, since it's a small and straightforward patch. i'll do it over the weekend if you don't beat me to it. > I used the -X flag because it wasn't taken, but I'm happy to rename it to a > better alternative. > i say you forget about a single-letter flag, and go with the new syntax, for example -R gc or --runcore=gc. ~jerry by the way, i've been thinking about adding a runcore with pre- and post-opcode hooks. a user could provide them in a library file that's linked in, allowing folks to write their own runcores without modifying any parrot core code (making maintenance of custom parrot builds easier.) it may also require or benefit from a registration system for runcores, so more could be added, but i'm not sure (especially at this point) that that's necessary. thoughts?
Re: Generic callback mechanism in NCI
[EMAIL PROTECTED] wrote: I started to write an OpenGL library and was only a couple of dozen lines into the pir when I remembered the documentation about callbacks in docs/pdds/draft/pdd16_native_call.pod . [...] My questions are: - Does anybody else want a generic callback function mechanism in NCI? - Is this a relatively sane/clean way to do it? - Is there a better way to generate the functions for each signature? - What is the right way to store that global user_data until the callbacks are fired? NCI isn't fully specified yet, but I'll outline some of our current thoughts and likely directions. Discussion welcome. We would like to eliminate the massive list of precompiled thunks for C function calls generated by call_list.txt. The tricky part is that you can't compile a C function at runtime. The best you can do is JIT it, and that depends on having a working JIT for the platform. We might be able to use LLVM's JIT, which would gain us a working JIT on a number of platforms. The JIT solution would continue to use the dlfunc interface, but instead of looking up a precompiled thunking function for the passed in signature, it would JIT one. As with the precompiled thunk, the JITed thunk is incorporated into an NCI sub object, which can be stored in a namespace, or passed around anonymously, and invoked, just like an ordinary sub object. That said, it's unlikely that we'll ever completely eliminate the list of precompiled thunks. Some platforms just won't have a JIT, and we can't afford to cut off NCI for the lack of a JIT. (For one thing, Parrot internals use NCI quite heavily.) But we can make them more manageable. We'll probably end up with something similar to src/ops/*.ops, with multiple files of signatures. The core file would be the absolute minimum required to run Parrot without loading external C libraries. Then we could add a file for each subsystem (MySQL, Postgres, pcre, SDL, Python builtins, and tcl, are a few already mentioned in call_list.txt). A configuration option when compiling Parrot could decide whether to precompile a restricted set, the full set, or add in some additional signatures for another external system. Some signature files for a particular library could be generated by a limited C parser, but it would always need to be checked by a human. Duplicate signatures between files would be merged. (And remember, this is only a fallback for platforms that can't JIT.) So, that's one part of the question. The other part is callbacks. If/when we develop a JIT solution for NCI call thunks, we can use the same technique for generating callback thunks. In the mean time, we'll have to continue with the precompiled callback thunks. Callbacks will use the concurrency scheduler. (There's some info about the concurrency scheduler in the new Events PDD, but more details will be in the Concurrency PDD next month.) For the moment all you need to know is that the concurrency scheduler is a central dispatcher in the Parrot interpreter that handles events, exceptions, async I/O, threads, etc. When you call 'new_callback', you pass it a Parrot sub, a user_data argument, and a signature. At the moment the signature is limited to two alternatives, in the future it will allow all the same signature options as an NCI call. The signature of the Parrot sub should match the signature passed into 'new_callback'. The 'new_callback' op will take these arguments and create a CallbackHandler PMC which stores the user data and the passed in sub, as well as any other information needed to properly invoke the right sub in the right interpreter. It then registers that CallbackHandler with the concurrency scheduler. Registering a callback handler returns a unique concurrency id (CID) for the handler, kind of like a process id. After registering the callback handler, 'new_callback' will look up a precompiled thunk or JIT a thunk with the requested signature. For the JITed thunk it will embed the CID as a constant within the thunk. For precompiled thunks, we're probably going to have to add more information to the CallbackHandler PMC (the signature, possibly a library identifier, etc). When the callback thunk is called from C, it bundles the C arguments into appropriate Parrot arguments, then notifies the concurrency scheduler that it has a callback with a particular CID, or particular set of characteristics (similar to scheduling an event of a particular type). The concurrency scheduler will look through its registered callback handlers to look for a matching handler, and if it finds one, invoke it, passing it the arguments that were passed to the C thunk. Essentially, this uses the concurrency scheduler as your global data store and as the source of unique identifiers. But, it's integrated with a core system. The immediate solution, to get OpenGL working now without waiting for the implementation of the concurrency scheduler an
[perl #46515] [BUG] t/pmc/threads.t test failures
The problem is specifically with vtable overrides in classes that are cloned into a thread (which is a cloned interpreter object). The sub object for the vtable override loses its 'seg' member, as demonstrated in the gdb output below. The particular vtable override in tests 13 and 14 of t/pmc/threads.t is 'get_string'. Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_PROTECTION_FAILURE at address: 0x0024 [Switching to process 26210 thread 0x2103] 0x0016f93e in Parrot_Sub_invoke (interp=0x1b17580, pmc=0x1cee0f8, next=0x0) at ./src/pmc/sub.pmc:231 231 pc = sub->seg->base.data + sub->start_offs; (gdb) bt #0 0x0016f93e in Parrot_Sub_invoke (interp=0x1b17580, pmc=0x1cee0f8, next=0x0) at ./src/pmc/sub.pmc:231 #1 0x000113c2 in runops_args (interp=0x1b17580, sub=0x1cee0f8, obj=0x1cecf88, meth_unused=0x1cf35e4, sig=0x24d200 "S", ap=0xb0182c54 "") at src/inter_run.c:185 #2 0x0001187a in Parrot_run_meth_fromc_args (interp=0x1b17580, sub=0x1cee0f8, obj=0x1cecf88, meth=0x1cf35e4, sig=0x24d200 "S") at src/inter_run.c:390 #3 0x001e29b0 in Parrot_Object_get_string (interp=0x1b17580, pmc=0x1cecf88) at ./src/pmc/object.c:2020 #4 0x0001646f in Parrot_oo_get_class (interp=0x1b17580, key=0x1cecf88) at src/oo.c:153 #5 0x00047f7e in Parrot_get_class_p_p (cur_opcode=0x2040ce4, interp=0x1b17580) at src/ops/object.ops:406 #6 0x000e4f1e in runops_slow_core (interp=0x1b17580, pc=0x2040ce4) at src/runops_cores.c:201 #7 0x000c4b53 in runops_int (interp=0x1b17580, offset=50) at src/interpreter.c:855 #8 0x00011266 in runops (interp=0x1b17580, offs=50) at src/inter_run.c:105 #9 0x00011508 in runops_args (interp=0x1b17580, sub=0x1ced3a8, obj=0x202dc48, meth_unused=0x0, sig=0x238314 "PF", ap=0xb0182e8c "H?\002\002") at src/inter_run.c:221 #10 0x00011648 in Parrot_runops_fromc_args (interp=0x1b17580, sub=0x1ced3a8, sig=0x238314 "PF") at src/inter_run.c:300 #11 0x0002d6cb in thread_func (arg=0x1ced468) at src/thread.c:426 #12 0x90024227 in _pthread_body () (gdb) print sub->seg $1 = (struct PackFile_ByteCode *) 0x0 (gdb)
[svn:parrot-pdd] r22275 - trunk/docs/pdds/draft
Author: kjs Date: Fri Oct 19 12:39:10 2007 New Revision: 22275 Modified: trunk/docs/pdds/draft/pdd06_pasm.pod Log: make line length test pass for pdd06_pasm.pod Modified: trunk/docs/pdds/draft/pdd06_pasm.pod == --- trunk/docs/pdds/draft/pdd06_pasm.pod(original) +++ trunk/docs/pdds/draft/pdd06_pasm.podFri Oct 19 12:39:10 2007 @@ -25,29 +25,31 @@ =over 4 =item * - Can we get rid of PASM ? - conversely, does PASM need to be kept up to date? - PASM is just a text form of PBC, so it should be kept - are there specific PBC features that can't currently be represented in PASM? - besides hll and :outer? -:init -lexicals? -:vtable -I'm a bit rusty, but anything that starts with a '.' or ':' is suspect - things that start with '.' are just directives to IMCC, equally applicable to PASM and PIR -isn't PASM separate from IMCC? - mdiep: it used to be separate -so to say that PASM can have directives is a major architectural change - perhaps the biggest thing we need is a definition of what PASM actually is - the line has grown quite fuzzy over the years - PASM could be defined as stringified PBC - compilable stringified pbc -it should be defined that way if we're going to call it assembly. - barney: that's the most likely direction, and if so, it has some implications for how PASM behaves - allison: which is what we want, anyway, right? - particle: yup - yes - good, looks like we're in agreement and headed in the proper direction on that topic. +Can we get rid of PASM ? + conversely, does PASM need to be kept up to date? + PASM is just a text form of PBC, so it should be kept + are there specific PBC features that can't currently be represented in PASM? + besides hll and :outer? + :init + lexicals? + :vtable + I'm a bit rusty, but anything that starts with a '.' or ':' is suspect + things that start with '.' are just directives to IMCC, +equally applicable to PASM and PIR + isn't PASM separate from IMCC? + mdiep: it used to be separate + so to say that PASM can have directives is a major architectural change + perhaps the biggest thing we need is a definition of what PASM actually is + the line has grown quite fuzzy over the years +PASM could be defined as stringified PBC + compilable stringified pbc + it should be defined that way if we're going to call it assembly. + barney: that's the most likely direction, and if so, it has some implications +for how PASM behaves + allison: which is what we want, anyway, right? + particle: yup +yes + good, looks like we're in agreement and headed in the proper direction on that topic. =back
[svn:parrot-pdd] r22279 - trunk/docs/pdds/draft
Author: kjs Date: Fri Oct 19 13:06:18 2007 New Revision: 22279 Modified: trunk/docs/pdds/draft/pdd06_pasm.pod Log: and yet another line too long in pdd06; fixing. Modified: trunk/docs/pdds/draft/pdd06_pasm.pod == --- trunk/docs/pdds/draft/pdd06_pasm.pod(original) +++ trunk/docs/pdds/draft/pdd06_pasm.podFri Oct 19 13:06:18 2007 @@ -1252,7 +1252,8 @@ =over 4 -=item * Fixed stack ops; push, pop, and clear properly documented according to the engine's behaviour now. +=item * Fixed stack ops; push, pop, and clear properly documented according to the +engine's behaviour now. =back
Re: [perl #46541] [BUG] t/pmc/objects.t test failures
chromatic wrote: On Friday 19 October 2007 09:30:31 Bernhard Schmalhofer wrote: Why should it be a TODO test? For one, there's no ticket for it. I saw the failing test, but didn't know where it came from or why it was there. (And until I dug into it the commit logs, I didn't even know if it was an old test that I had broken while working on the threads tests.) TODOing known bugs for a release is a different issue. I don't see why. I'd like to see Parrot be in a releasable state with every checkin. That's why we have the tests. I support this as a rule. I broke it when I merged in the pdd15oo branch with 4 failing tests because I was planning to fix them right away. I did cut it down to 3 failing tests, and resolved the surface issues with the remaining 3. But, there's a deeper issue with cloning vtable overrides across threads that I haven't resolved yet (the interpreter cloning code is sorely in need of a rewrite). So, I'll mark those as TODO, with a reference to the tickets that James created (thanks, James!). Allison
Re: [svn:parrot-pdd] r22268 - trunk/docs/pdds/draft
It seems as though the first line seems to have caught my eye.
In most languages, ~ is used for bitwise not, and not bitwise xor which
is given ^. Parrot seems to do things a tad differently. An analysis
of the generated pir shows how parrot treats it now.
.sub main
$I0 = 5
$I1 = 3
$I2 = ~$I0
$I3 = $I0 ~ $I1
$I4 = $I0 ~~ $I1
.end
main:
set I0, 5
set I1, 3
bnot I2, I0
bxor I3, I0, I1
xor I4, I0, I1
set_returns
returncc
Not, `bxor $I0, 0` is the same as `bnot $I0` so it's logical for that,
but it still seems odd that parrot's different. Is there a reason why
parrot's using ~ instead of ^ for xor?
On Oct 19, 2007, at 8:21 AM, [EMAIL PROTECTED] wrote:
Author: ambs
Date: Fri Oct 19 06:21:28 2007
New Revision: 22268
Modified:
trunk/docs/pdds/draft/pdd19_pir.pod
Log:
Added proposal for simple change on binary operators
Modified: trunk/docs/pdds/draft/pdd19_pir.pod
===
===
--- trunk/docs/pdds/draft/pdd19_pir.pod (original)
+++ trunk/docs/pdds/draft/pdd19_pir.pod Fri Oct 19 06:21:28 2007
@@ -534,6 +534,9 @@
B<&>, B<|> and B<~> are binary B, B and B.
+{{PROPOSAL: Change description to support logic operators
(comparisons) as
+implemented (and working) in imcc.y.}}
+
=item =
This is equivalent to
Re: [perl #46541] [BUG] t/pmc/objects.t test failures
chromatic schrieb: On Thursday 18 October 2007 18:04:15 James Keenan wrote: Linux. r22261 [li11-226:parrot] 559 $ prove -v t/pmc/objects.t t/pmc/objects1..74 ok 1 - find_type (base class) [snip] not ok 57 - equality of subclassed Integer # Failed test (t/pmc/objects.t at line 1668) # got: '123 is not equal to 123 # ' # expected: '123 is equal to 123 # ' ok 58 - short name attributes Bernhard added this one in r22241, but didn't TODO it or file a ticket (as I can tell). TODOd in 22265. Why should it be a TODO test? As far as I see, it exposes a real bug. So I want a red light going on. TODOing known bugs for a release is a different issue. Regards, Bernhard
Re: [perl #43481] t/examples/shootout.t (shootout_16.pir) fails on gentoo/x86
> I'm not sure about the current status of t/examples/shootout.t on Linux, > but it is still failing on Darwin. See attached. > It just passed on my Mandriva. -- Email and shopping with the feelgood factor! 55% of income to good causes. http://www.ippimail.com
Re: [perl #46541] [BUG] t/pmc/objects.t test failures
On Friday 19 October 2007 11:37:08 Bernhard Schmalhofer wrote: > chromatic schrieb: > > For two, allowing known failing bugs is the Broken Windows anti-pattern. > > Not only does it undermine confidence in the software (oh, still broken!) > > but it adds noise that makes it far more difficult to know if a change > > introduced a new bug (which tests were failing? 1, 3, and... oh forget > > it!) > Would Smolder address the question: Which tests were introduced by my > last change ? Perhaps. > I am missing here the distinction between: > A) There is an unimplemented feature so I add TODO tests for that > B) This feature should be working, but it isn't. > C) It is known what the problem is, it just isn't fixed yet The distinction is between: A) My most recent change did not break anything (all tests pass) B) My most recent change broke something (some tests failed) Any failing tests that stay failing for more than a checkin or two break that model. Yes, that includes the two or three remaining failures from the PDD 15 merge back, and yes we should TODO those. Allison is working on them, so she knows what they are and can unTODO them when she fixes them. -- c
[svn:parrot-pdd] r22268 - trunk/docs/pdds/draft
Author: ambs
Date: Fri Oct 19 06:21:28 2007
New Revision: 22268
Modified:
trunk/docs/pdds/draft/pdd19_pir.pod
Log:
Added proposal for simple change on binary operators
Modified: trunk/docs/pdds/draft/pdd19_pir.pod
==
--- trunk/docs/pdds/draft/pdd19_pir.pod (original)
+++ trunk/docs/pdds/draft/pdd19_pir.pod Fri Oct 19 06:21:28 2007
@@ -534,6 +534,9 @@
B<&>, B<|> and B<~> are binary B, B and B.
+{{PROPOSAL: Change description to support logic operators (comparisons) as
+implemented (and working) in imcc.y.}}
+
=item =
This is equivalent to
