Greetings,
I've recently been spending some quality time with the plpython module, and I
think I'm well on the road to an improved version of it(although, nothing about a
trusted variant). By improved, I mostly mean cleaned up, and reorganized..
Here are some of the changes that I have made in my own version:
Compilation and execution have been greatly simplified and should be faster(at
least execution should be).
Caching of compiled code no longer references a Python
dictionary(PLyProcedureCache). The handler keeps its own vector of procedure
structs(should be faster, and is trivial).
Removal of plpython generated dictionaries SD and GD. They don't seem be very
useful, as they are forgotten when the postmaster exits and not remembered when a new
one starts. SD is questionable, does/did anyone find SD very useful? GD seems almost
pointless as the global keyword should be sufficient. Although, I do think there was a
mention of GD being "safe globals", but I don't know why it would be safer than
"global var".
Removal of the built-in "plpy" python module that plpython creates. This is
done because it provides interfaces to pgsql functions that I feel should be located
elsewhere; elsewhere being another python module. I've already generated a preliminary
interface to elog and SPI_* with SWIG that at first glance seems quite functional(it
links, and is at least able to properly call elog, I haven't really tested SPI).
Improvement to tracebacks, as it now NOTICE's the python tracebacks(There is
already an ERROR, so I don't think WARNING is necessary). PLy_traceback, originally,
seemed to ignore the tb of the PyErr_Fetch.
Removal of plpython type conversion routines and data structures. This was
done because I felt that there was a better way to do it. Not sure what yet, as it is
one of my questions to the list, but it will probably end up being a similar
implementation.
I also plan to make some changes to trigger handling, but I haven't done
anything worth mentioning yet..
Type conversion
plpython's current type conversion implementation appears to be dependent on
strings as the common format. This is fine, but not very extensible as is, unless you
don't mind explicitly parsing strings inside each function that takes an unsupported
data type.
I was thinking that a better solution would be creating a python object type
inside the database. Thus allowing users to write casts to and from non-standard or
unimplemented data types with little difficulty(well, maybe some :). This would allow
conversion in an extensible way, which doesn't require modification to plpython.
Storage could be easily achieved by pickling the object.
Another thought would be to just pass valid PyObject pointers in and out of
conversion procedures, effectively disallowing storage(outside the process in which
the object was created in), unless it is possible to have a persistent storage
mechanism that makes it possible to go through pickle?.?..(yeah, I'm new to pgsql dev).
Python PostgreSQL Interface
plpython, currently, implements its own built-in module to interface with a
few pgsql routines, and it works, but I feel it should be located elsewhere, as I said
before.
For the most part, I can only see most people using elog, and SPI within plpy,
but perhaps that is too narrow of a view. Perhaps it would be useful to many to have
access to some backend routines through plpy, but I'm not sure and that is why I'm
asking the list.
How far should such an PostgreSQL interface module go?
What should its name be if full/semi-full interface is created? I was thinking
simply py-pgsql as the package name, and the module name, of course, would be pgsql.
What should the name be if it was only elog and SPI? py-pgspi?
I'm leaning towards py-pgsql, a partial interface consisting of elog and SPI
and perhaps a few other useful routines. But have the module as a package as to allow
easy extensions to the package as subpackages..
From this interface, a DB-API 2.0 compatible SPI interface will come as well.
My version has a short ways to go before it is ready for usage, but if you
want to see what I've done, just drop me an e-mail.
Comments? Criticisms? Feature suggestions?
Anyone else doing significant work on plpython?
-James
pgp00000.pgp
Description: PGP signature
