Re: [Python-Dev] [Python-checkins] r45925 - in python/trunk: Lib/tempfile.py Lib/test/test_os.py Misc/NEWS Modules/posixmodule.c
Martin v. Löwis wrote:
> M.-A. Lemburg wrote:
>> What do you think ?
>
> I think the size could be further reduced by restricting the set of
> error codes. For example, if the COM error codes are left out, I only
> get a Python file with 60k source size (although the bytecode size
> is then 130k). I'm not sure whether GetLastError can ever return a COM
> error code, but in my experience, it never did.
I was leaving those out already - only the codes named 'ERROR_*'
get included (see attached parser and generator).
> I wouldn't want to introduce a clumsy interface just to achieve space
> savings. If people consider the space consumption of 200k disk
> size unacceptable (or would never use the module because of the runtime
> size effects), the entire idea should be dropped (IMO). OTOH, I don't
> find such a size increase unacceptable myself.
Using a lookup object is not really clumsy - you can still access
all the values by attribute access. The only difference is that
they don't live in the module namespace, but get accessed via
an object.
I'm not worried about the disk space being used. The heap
memory usage is what's worrying: the import of the module lets
the non-shared memory size of the process grow by 700kB
on my AMD64 box.
Regards,
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, May 13 2006)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free !
#!/usr/bin/env python
""" Create the winerror module from the data found in the
Windows Platform SDK WinError.h file.
"""
import re
WINERROR_H = './WinError.h'
MODULE_LAYOUT = """\
#
# winerror module
#
# Generated from the Windows Platform SDK include file WinError.h
#
# Error symbols
%s
# Error codes dictionary
%s
"""
ERROR_DEF_RE = re.compile('#define\s+(ERROR_[A-Z0-9_]+)\s+([0-9]+)')
def parse_defs(filename):
file = open(filename)
d = {}
for line in file:
m = ERROR_DEF_RE.match(line)
if m is None:
continue
symbol, value = m.groups()
d[symbol] = int(value)
return d
def generate_python_dict_literal(dictname, data):
items = data.items()
items.sort()
maxkeylength = max([len(repr(key))
for key in data.iterkeys()])
formatstring = ' %%-%ir: %%r,' % maxkeylength
l = ['%s = {' % dictname]
l.extend([formatstring % (key, value)
for (key, value) in items])
l.append('}')
return '\n'.join(l)
def invdict(d):
return dict([(value, key) for (key, value) in d.iteritems()])
def generate_python_attribute_definitions(data):
items = data.items()
items.sort()
maxkeylength = max([len(repr(key))
for key in data.iterkeys()])
formatstring = '%%-%is = %%r' % maxkeylength
l = [formatstring % (key, value)
for (key, value) in items]
return '\n'.join(l)
def generate_winerror_module(data):
return MODULE_LAYOUT % (
generate_python_attribute_definitions(data),
generate_python_dict_literal('errorcode', invdict(data)))
if __name__ == '__main__':
data = parse_defs(WINERROR_H)
print generate_winerror_module(data)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r45925 - in python/trunk: Lib/tempfile.py Lib/test/test_os.py Misc/NEWS Modules/posixmodule.c
M.-A. Lemburg wrote: > I was leaving those out already - only the codes named 'ERROR_*' > get included (see attached parser and generator). Right. One might debate whether DNS_INFO_AXFR_COMPLETE (9751L) or WSAEACCES (10013L) should be included as well. I got a smaller source file as I included only forward mappings, and used a loop to create the backwards mappings. > Using a lookup object is not really clumsy - you can still access > all the values by attribute access. The only difference is that > they don't live in the module namespace, but get accessed via > an object. So how much space would that save? > I'm not worried about the disk space being used. The heap > memory usage is what's worrying: the import of the module lets > the non-shared memory size of the process grow by 700kB > on my AMD64 box. That number must be misleading somehow. There are 1510 strings, with a total length of 39972. There are 1510 integers also, and they all get added into three dictionaries. On a 32-bit machine, these should consume 76968 bytes for the strings (*), 18120 bytes for the integers, and 10 bytes for the dict entries (**), for a total of 20 bytes at run-time. On a 64-bit machine, the strings should consume 101128 bytes (***), the integers 24160, and the dict entries 20 bytes, for a total of 325000 bytes. >From that, I would conclude that one should avoid 64-bit machines if one is worried about memory usage :-) Regards, Martin (*) assuming 20 bytes string header, 1 byte null-termination, and a rounding-up to the next multiple of 8 (**) assuming 12 bytes per dict entry in three dictionaries (winerror.__dict__, winerror.errorcode, interning dict), and assuming an average fill ratio of the dicts of 50% (***) assuming 40 bytes string header, provided long is a 64-bit type on that platform ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Py_ssize_t formatting
Hi, which formatting code should be used for a Py_ssize_t value in e.g. PyString_FromFormat? '%zd' won't work since my value can be negative, and the 'z' modifier converts the argument to size_t. Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_ssize_t formatting
Georg Brandl wrote: > which formatting code should be used for a Py_ssize_t value in e.g. > PyString_FromFormat? %zd > '%zd' won't work since my value can be negative, and the 'z' modifier > converts the argument to size_t. That's a bug. It should print it signed. If unsigned printing of size_t is desired, %zu should be used. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_ssize_t formatting
Martin v. Löwis wrote: > Georg Brandl wrote: >> which formatting code should be used for a Py_ssize_t value in e.g. >> PyString_FromFormat? > > %zd > >> '%zd' won't work since my value can be negative, and the 'z' modifier >> converts the argument to size_t. > > That's a bug. It should print it signed. If unsigned printing of size_t > is desired, %zu should be used. I saw you fixed it, thanks. Similary, there's no way for a structmember to be of type Py_ssize_t... Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_ssize_t formatting
Georg Brandl wrote: > Similary, there's no way for a structmember to be of type Py_ssize_t... Right. At least, not with changing structmember.[ch]. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] cleaned windows icons
Somebody has contributed an edited version of the current icons, at https://sourceforge.net/tracker/index.php?func=detail&aid=1481304&group_id=5470&atid=305470 He claims that his version is prettier in small sizes; I'm uncertain whether this is the case. What do you think? Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_ssize_t formatting
Martin v. Löwis wrote: > Georg Brandl wrote: >> Similary, there's no way for a structmember to be of type Py_ssize_t... > > Right. At least, not with changing structmember.[ch]. Did you mean "without"? Can I submit a patch? Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_ssize_t formatting
On 5/13/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
>
> > '%zd' won't work since my value can be negative, and the 'z' modifier
> > converts the argument to size_t.
>
> That's a bug. It should print it signed. If unsigned printing of size_t
> is desired, %zu should be used.
Looking in stringobject.c, I don't see how %zu (or %lu) can be used
with String_FromFormatV. Here's the code around line 260 (note
ssize_t in the comment really means Py_ssize_t):
/* handle the long flag, but only for %ld. others
can be added when necessary. */
if (*f == 'l' && *(f+1) == 'd') {
longflag = 1;
++f;
}
/* handle the ssize_t flag. */
if (*f == 'z' && *(f+1) == 'd') {
size_tflag = 1;
++f;
}
n
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_ssize_t formatting
Neal Norwitz wrote: >> That's a bug. It should print it signed. If unsigned printing of size_t >> is desired, %zu should be used. > > Looking in stringobject.c, I don't see how %zu (or %lu) can be used > with String_FromFormatV. Right. It currently cannot be used. So if it is desired, it needs to be added first, and then should be used. Looking at the checkin messages, I understand that the change from "d" to "u" was to make some compiler stop warning - that is the wrong motivation for such a behavioral change. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_ssize_t formatting
[Neal] >> Looking in stringobject.c, I don't see how %zu (or %lu) can be used >> with String_FromFormatV. [Martin] > Right. It currently cannot be used. So if it is desired, it needs to > be added first, and then should be used. I added it: %u, %lu, and %zu can be used now in PyString_FromFormat, PyErr_Format, and PyString_FromFormatV. Since PyString_FromFormat and PyErr_Format have exactly the same rules (both inherited from PyString_FromFormatV), but their docs were way out of synch, it would be good if someone with more LaTeX Fu changed one of them to just point to the other. I simply did a massive copy+paste job to get them in synch again. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py_ssize_t formatting
Georg Brandl wrote: >> Right. At least, not with changing structmember.[ch]. > > Did you mean "without"? Oops, right. > Can I submit a patch? I personally don't mind having more types added to structmember, so I'm +0 on adding Py_ssize_t to the list of types supported. I wonder what the specific application is that you have in mind, though. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
