Re: [Python-Dev] __str__ vs. __unicode__
Walter Dörwald wrote: M.-A. Lemburg wrote: > [...] __str__ and __unicode__ as well as the other hooks were specifically added for the type constructors to use. However, these were added at a time where sub-classing of types was not possible, so it's time now to reconsider whether this functionality should be extended to sub-classes as well. So can we reach consensus on this, or do we need a BDFL pronouncement? I don't have a clear picture of what the consensus currently looks like :-) If we're going for for a solution that implements the hook awareness for all hooks, I'd be +1 on that. If we only touch the __unicode__ case, we'd only be created yet another special case. I'd vote -0 on that. Another solution would be to have all type constructors ignore the hooks (which were originally added to provide classes with a way to mimic type behavior). In general, I think we should try to get rid off special cases and go for a clean solution (either way). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jan 23 2005) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Speed up function calls
I added a patch to SF: http://python.org/sf/1107887
I would like feedback on whether the approach is desirable.
The patch adds a new method type (flags) METH_ARGS that is used in
PyMethodDef. METH_ARGS means the min and max # of arguments are
specified in the PyMethodDef by adding 2 new fields. This information
can be used in ceval to
call the method. No tuple packing/unpacking is required since the C
stack is used.
The benefits are:
* faster function calls
* simplify function call machinery by removing METH_NOARGS, METH_O,
and possibly METH_VARARGS
* more introspection info for C functions (ie, min/max arg count)
(not implemented)
The drawbacks are:
* the defn of the MethodDef (# args) is separate from the function defn
* potentially more error prone to write C methods???
I've measured between 13-22% speed improvement (debug build on
Operton) when doing simple tests like:
./python ./Lib/timeit.py -v 'pow(3, 5)'
I think the difference tends to be fairly constant at about .3 usec per loop.
Here's a portion of the patch to show the difference between conventions:
-builtin_filter(PyObject *self, PyObject *args)
+builtin_filter(PyObject *self, PyObject *func, PyObject *seq)
{
- PyObject *func, *seq, *result, *it, *arg;
+ PyObject *result, *it, *arg;
int len; /* guess for result list size */
register int j;
- if (!PyArg_UnpackTuple(args, "filter", 2, 2, &func, &seq))
- return NULL;
-
# the are no other changes between METH_O and METH_ARGS
- {"abs", builtin_abs,METH_O, abs_doc},
+ {"abs", builtin_abs,METH_ARGS, abs_doc, 1, 1},
- {"filter", builtin_filter, METH_VARARGS, filter_doc},
+ {"filter", builtin_filter, METH_ARGS, filter_doc, 2, 2},
Neal
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Re: Speed up function calls
Neal Norwitz wrote: > The patch adds a new method type (flags) METH_ARGS that is used in > PyMethodDef. METH_ARGS means the min and max # of arguments are > specified in the PyMethodDef by adding 2 new fields. > * the defn of the MethodDef (# args) is separate from the function defn > * potentially more error prone to write C methods??? "potentially"? sounds like a recipe for disaster. but the patch is nice, and more speed never hurts. maybe it's time to write that module fixup preprocessor thing that Guido should have written some 15 years ago... ;-) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Weekly Python Patch/Bug Summary
Patch / Bug Summary ___ Patches : 273 open ( +1) / 2746 closed ( +9) / 3019 total (+10) Bugs: 797 open ( +4) / 4789 closed (+12) / 5586 total (+16) RFE : 166 open ( +1) / 141 closed ( +0) / 307 total ( +1) New / Reopened Patches __ fix distutils.install.dump_dirs() with negated options (2005-01-17) CLOSED http://python.org/sf/1103844 opened by Wummel Add O_SHLOCK/O_EXLOCK to posix (2005-01-17) http://python.org/sf/1103951 opened by Skip Montanaro setup.py --help and --help-commands altered. (2005-01-17) http://python.org/sf/1104111 opened by Titus Brown new-style exceptions (2005-01-18) http://python.org/sf/1104669 opened by Michael Hudson misc doc typos (2005-01-18) CLOSED http://python.org/sf/1104868 opened by DSM chr, ord, unichr documentation updates (2004-10-31) http://python.org/sf/1057588 reopened by mike_j_brown Faster commonprefix in macpath, ntpath, etc. (2005-01-19) http://python.org/sf/1105730 opened by Jimmy Retzlaff get rid of unbound methods (mostly) (2005-01-17) CLOSED http://python.org/sf/1103689 opened by Guido van Rossum Updated "Working on Cygwin" section (2005-01-22) http://python.org/sf/1107221 opened by Alan Green Add Thread.isActive() (2005-01-23) http://python.org/sf/1107656 opened by Alan Green Speed up function calls/can add more introspection info (2005-01-23) http://python.org/sf/1107887 opened by Neal Norwitz Patches Closed __ fix distutils.install.dump_dirs() with negated options (2005-01-17) http://python.org/sf/1103844 closed by theller ast-branch: fix for coredump from new import grammar (2005-01-11) http://python.org/sf/1100563 closed by kbk Shadow Password Support Module (2002-07-10) http://python.org/sf/579435 closed by loewis misc doc typos (2005-01-18) http://python.org/sf/1104868 closed by fdrake extending readline functionality (2003-02-11) http://python.org/sf/684500 closed by fdrake self.button.pack() in tkinter.tex example (2005-01-03) http://python.org/sf/1094815 closed by fdrake Clean up discussion of new C thread idiom (2004-09-20) http://python.org/sf/1031233 closed by fdrake Description of args to IMAP4.store() in imaplib (2004-12-12) http://python.org/sf/1084092 closed by fdrake get rid of unbound methods (mostly) (2005-01-17) http://python.org/sf/1103689 closed by gvanrossum New / Reopened Bugs ___ email.base64MIME.header_encode vs RFC 1522 (2005-01-17) http://python.org/sf/1103926 opened by Ucho wishlist: os.feed_urandom(input) (2005-01-17) http://python.org/sf/1104021 opened by Zooko O'Whielacronx configure doesn't set up CFLAGS properly (2005-01-17) http://python.org/sf/1104249 opened by Bryan O'Sullivan Bugs in _csv module - lineterminator (2004-11-24) http://python.org/sf/1072404 reopened by fresh Wrong expression with \w+? (2005-01-18) CLOSED http://python.org/sf/1104608 opened by rengel Bug in String rstrip method (2005-01-18) CLOSED http://python.org/sf/1104923 opened by Rick Coupland Undocumented implicit strip() in split(None) string method (2005-01-19) http://python.org/sf/1105286 opened by YoHell Warnings in Python.h with gcc 4.0.0 (2005-01-19) http://python.org/sf/1105699 opened by Bob Ippolito incorrect constant names in curses window objects page (2005-01-19) http://python.org/sf/1105706 opened by dcrosta null source chars handled oddly (2005-01-19) http://python.org/sf/1105770 opened by Reginald B. Charney bug with idle's stdout when executing load_source (2005-01-20) http://python.org/sf/1105950 opened by imperialfists os.stat int/float oddity (2005-01-20) CLOSED http://python.org/sf/1105998 opened by George Yoshida README of 2.4 source download says 2.4a3 (2005-01-20) http://python.org/sf/1106057 opened by Roger Erens semaphore errors from Python 2.3.x on AIX 5.2 (2005-01-20) http://python.org/sf/1106262 opened by The Written Word slightly easier way to debug from the exception handler (2005-01-20) http://python.org/sf/1106316 opened by Leonardo Rochael Almeida os.makedirs() ignores mode parameter (2005-01-21) http://python.org/sf/1106572 opened by Andreas Jung split() takes no keyword arguments (2005-01-21) http://python.org/sf/1106694 opened by Vinz os.pathsep is wrong on Mac OS X (2005-01-22) CLOSED http://python.org/sf/1107258 opened by Mac-arena the Bored Zo Bugs Closed ___ --without-cxx flag of configure isn't documented. (2003-03-12) http://python.org/sf/702147 closed by bcannon presentation typo in lib: 6.21.4.2 How callbacks are called (2004-12-22) http://python.org/sf/1090139 closed by gward rfc822 Deprecated since release 2.3? (2005-01-15) h
[Python-Dev] Microsoft to Provide PyCon Opening Keynote
Dear Python Colleague: The PyCon Program Committee is happy to announce that the opening keynote speech, at 9:30 am on Wednesday March 23 will be: Python on the .NET Platform, by Jim Hugunin, Microsoft Corporation Jim Hugunin is well-known in the Python world for his pioneering work on JPython (now Jython), and more recently for the IronPython .NET implementation of Python. Jim joined Microsoft's Common Language Runtime team in August last year to continue his work on Iron Python and further improve the CLR's support for dynamic languages like Python. I look forward to hearing what Jim has to say, and hope that you will join me and the rest of the Python community at PyCon DC 2005, at George Washington University from March 23-25, with a four-day sprint starting on Saturday March 19. Early bird registration rates are still available for a few more days. Go to http://www.python.org/moin/PyConDC2005/Schedule for the current schedule, and register at http://www.python.org/pycon/2005/ regards Steve Holden Chairman, PyCON DC 2005 -- PyCon DC 2005: The third Python Community Conference http://www.pycon.org/ http://www.python.org/pycon/ The scoop on Python implementations and applications ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Improving the Python Memory Allocator
This message is a follow up to a thread I started on python-dev back in October, archived here: http://mail.python.org/pipermail/python-dev/2004-October/049480.html Basically, the problem I am trying to solve is that the Python memory allocator never frees memory back to the operating system. I have attached a patch against obmalloc.c for discussion. The patch still has some rough edges and possibly some bugs, so I don't think it should be merged as is. However, I would appreciate any feedback on the chances for getting this implementation into the core. The rest of this message lists some disadvantages to this implementation, a description of the important changes, a benchmark, and my future plans if this change gets accepted. The patch works for any version of Python that uses obmalloc.c (which includes Python 2.3 and 2.4), but I did my testing with Python 2.5 from CVS under Linux and Mac OS X. This version of the allocator will actually free memory. It has two disadvantages: First, there is slightly more overhead with programs that allocate a lot of memory, release it, then reallocate it. The original allocator simply holds on to all the memory, allowing it to be efficiently reused. This allocator will call free(), so it also must call malloc() again when the memory is needed. I have a "worst case" benchmark which shows that this cost isn't too significant, but it could be a problem for some workloads. If it is, I have an idea for how to work around it. Second, the previous allocator went out of its way to permit a module to call PyObject_Free while another thread is executing PyObject_Malloc. Apparently, this was a backwards compatibility hack for old Python modules which erroneously call these functions without holding the GIL. These modules will have to be fixed if this implementation is accepted into the core. Summary of the changes: - Add an "arena_object" structure for tracking pages that belong to each 256kB arena. - Change the "arenas" array from an array of pointers to an array of arena_object structures. - When freeing a page (a pool), it is placed on a free pool list for the arena it belongs to, instead of a global free pool list. - When freeing a page, if the arena is completely unused, the arena is deallocated. - When allocating a page, it is taken from the arena that is the most full. This gives arenas that are almost completely unused a chance to be freed. Benchmark: The only benchmark I have performed at the moment is the worst case for this allocator: A program that allocates 1 000 000 Python objects which occupy nearly 200MB, frees them, reallocates them, then quits. I ran the program four times, and discarded the initial time. Here is the object: class Obj: def __init__( self ): self.dumb = "hello" And here are the average execution times for this program: Python 2.5: real time: 16.304 user time: 16.016 system: 0.257 Python 2.5 + patch: real time: 16.062 user time: 15.593 system: 0.450 As expected, the patched version spends nearly twice as much system time than the original version. This is because it calls free() and malloc() twice as many times. However, this difference is offset by the fact that the user space execution time is actually *less* than the original version. How is this possible? The likely cause is because the original version defined the arenas pointer to be "volatile" in order to work when Free and Malloc were called simultaneously. Since this version breaks that, the pointer no longer needs to be volatile, which allows the value to be stored in a register instead of being read from memory on each operation. Here are some graphs of the memory allocator behaviour running this benchmark. Original: http://www.eng.uwaterloo.ca/~ejones/original.png New: http://www.eng.uwaterloo.ca/~ejones/new.png Future Plans: - More detailed benchmarking. - The "specialized" allocators for the basic types, such as ints, also need to free memory back to the system. - Potentially the allocator should keep some amount of free memory around to improve the performance of programs that cyclically allocate and free large amounts of memory. This amount should be "self-tuned" to the application. Thank you for your feedback, Evan Jones python-allocator.diff Description: Binary data ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
