[EMAIL PROTECTED] wrote:
I started to write an OpenGL library and was only a couple of dozen
lines into the pir when I remembered the documentation about callbacks
in docs/pdds/draft/pdd16_native_call.pod .
[...]
My questions are:
- Does anybody else want a generic callback function mechanism in NCI?
- Is this a relatively sane/clean way to do it?
- Is there a better way to generate the functions for each signature?
- What is the right way to store that global user_data until the callbacks
are fired?
NCI isn't fully specified yet, but I'll outline some of our current
thoughts and likely directions. Discussion welcome.
We would like to eliminate the massive list of precompiled thunks for C
function calls generated by call_list.txt. The tricky part is that you
can't compile a C function at runtime. The best you can do is JIT it,
and that depends on having a working JIT for the platform. We might be
able to use LLVM's JIT, which would gain us a working JIT on a number of
platforms.
The JIT solution would continue to use the dlfunc interface, but instead
of looking up a precompiled thunking function for the passed in
signature, it would JIT one. As with the precompiled thunk, the JITed
thunk is incorporated into an NCI sub object, which can be stored in a
namespace, or passed around anonymously, and invoked, just like an
ordinary sub object.
That said, it's unlikely that we'll ever completely eliminate the list
of precompiled thunks. Some platforms just won't have a JIT, and we
can't afford to cut off NCI for the lack of a JIT. (For one thing,
Parrot internals use NCI quite heavily.) But we can make them more
manageable. We'll probably end up with something similar to
src/ops/*.ops, with multiple files of signatures. The core file would be
the absolute minimum required to run Parrot without loading external C
libraries. Then we could add a file for each subsystem (MySQL, Postgres,
pcre, SDL, Python builtins, and tcl, are a few already mentioned in
call_list.txt). A configuration option when compiling Parrot could
decide whether to precompile a restricted set, the full set, or add in
some additional signatures for another external system. Some signature
files for a particular library could be generated by a limited C parser,
but it would always need to be checked by a human. Duplicate signatures
between files would be merged. (And remember, this is only a fallback
for platforms that can't JIT.)
So, that's one part of the question. The other part is callbacks.
If/when we develop a JIT solution for NCI call thunks, we can use the
same technique for generating callback thunks. In the mean time, we'll
have to continue with the precompiled callback thunks.
Callbacks will use the concurrency scheduler. (There's some info about
the concurrency scheduler in the new Events PDD, but more details will
be in the Concurrency PDD next month.) For the moment all you need to
know is that the concurrency scheduler is a central dispatcher in the
Parrot interpreter that handles events, exceptions, async I/O, threads, etc.
When you call 'new_callback', you pass it a Parrot sub, a user_data
argument, and a signature. At the moment the signature is limited to two
alternatives, in the future it will allow all the same signature options
as an NCI call. The signature of the Parrot sub should match the
signature passed into 'new_callback'.
The 'new_callback' op will take these arguments and create a
CallbackHandler PMC which stores the user data and the passed in sub, as
well as any other information needed to properly invoke the right sub in
the right interpreter. It then registers that CallbackHandler with the
concurrency scheduler. Registering a callback handler returns a unique
concurrency id (CID) for the handler, kind of like a process id.
After registering the callback handler, 'new_callback' will look up a
precompiled thunk or JIT a thunk with the requested signature. For the
JITed thunk it will embed the CID as a constant within the thunk. For
precompiled thunks, we're probably going to have to add more information
to the CallbackHandler PMC (the signature, possibly a library
identifier, etc).
When the callback thunk is called from C, it bundles the C arguments
into appropriate Parrot arguments, then notifies the concurrency
scheduler that it has a callback with a particular CID, or particular
set of characteristics (similar to scheduling an event of a particular
type). The concurrency scheduler will look through its registered
callback handlers to look for a matching handler, and if it finds one,
invoke it, passing it the arguments that were passed to the C thunk.
Essentially, this uses the concurrency scheduler as your global data
store and as the source of unique identifiers. But, it's integrated with
a core system.
The immediate solution, to get OpenGL working now without waiting for
the implementation of the concurrency scheduler and JITed call/callback
thunks, is to add a few more callback signatures to the current set of
alternatives, and to write a little bit of custom C code for the cases
that can't pass dispatch information in a user data argument.
I wouldn't go so far as implementing the callback_list.txt option at
this point. You'll have at least a basic implementation of the
concurrency scheduler by December 1st as we need it for events.
Allison
