Python 3.2.2rc1

[Martin v. Löwis]

On behalf of the Python development team, I'm happy to announce the
first release candidate of the Python 3.2.2 maintenance release (3.2.2rc1).

Python 3.2.2 fixes `a regression <http://bugs.python.org/12576>`_ in the
``urllib.request`` module that prevented opening many HTTP resources
correctly
with Python 3.2.1.

Python 3.2 is a continuation of the efforts to improve and stabilize the
Python 3.x line.  Since the final release of Python 2.7, the 2.x line
will only receive bugfixes, and new features are developed for 3.x only.

Since PEP 3003, the Moratorium on Language Changes, is in effect, there
are no changes in Python's syntax and built-in types in Python 3.2.
Development efforts concentrated on the standard library and support for
porting code to Python 3.  Highlights are:

* numerous improvements to the unittest module
* PEP 3147, support for .pyc repository directories
* PEP 3149, support for version tagged dynamic libraries
* PEP 3148, a new futures library for concurrent programming
* PEP 384, a stable ABI for extension modules
* PEP 391, dictionary-based logging configuration
* an overhauled GIL implementation that reduces contention
* an extended email package that handles bytes messages
* a much improved ssl module with support for SSL contexts and certificate
  hostname matching
* a sysconfig module to access configuration information
* additions to the shutil module, among them archive file support
* many enhancements to configparser, among them mapping protocol support
* improvements to pdb, the Python debugger
* countless fixes regarding bytes/string issues; among them full support
  for a bytes environment (filenames, environment variables)
* many consistency and behavior fixes for numeric operations

For a more extensive list of changes in 3.2, see

http://docs.python.org/3.2/whatsnew/3.2.html

To download Python 3.2 visit:

http://www.python.org/download/releases/3.2/

Please consider trying Python 3.2 with your code and reporting any bugs
you may notice to:

http://bugs.python.org/


Enjoy!

--
Martin v. Löwis
(on behalf of the entire python-dev team and 3.2's contributors)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: 2to3 and maketrans

[Martin v. Löwis]

Am 04.03.2011 03:21, schrieb Dan Stromberg:
> 
> On Thu, Mar 3, 2011 at 3:46 PM, Martin v. Loewis  > wrote:
> 
> That depends on how you chose to represent text in 2.7.
> The recommended way for that (also with 3.x in mind)
> is that you should use Unicode strings to represent text.
> 
> 
> For application programming, I'm sure Unicode is usually preferable.
> 
> For systems programming, it's hard for me to imagine that unicode would
> normally be favored over bytes.

I think Greg Ewing was really talking about WxPython. For WxPython,
I would indeed recommend to represent text as Unicode, although
there may be backwards-compatibility concerns to support byte strings
in certain places as well.

Whether a GUI library is application programming or systems programming,
I don't know.

Regards,
Martin

-- 
http://mail.python.org/mailman/listinfo/python-list

[ANN] Python 2.5.6 Release Candidate 1

[Martin v. Löwis]

On behalf of the Python development team and the Python community, I'm
happy to announce the release candidate 1 of Python 2.5.6.

This is a source-only release that only includes security fixes. The
last full bug-fix release of Python 2.5 was Python 2.5.4. Users are
encouraged to upgrade to the latest release of Python 2.7 (which is
2.7.1 at this point).

This releases fixes issues with the urllib, urllib2, SimpleHTTPServer,
and audiop modules. See the release notes at the website (also
available as Misc/NEWS in the source distribution) for details of bugs
fixed.

For more information on Python 2.5.6, including download links for
various platforms, release notes, and known issues, please see:

http://www.python.org/2.5.6

Highlights of the previous major Python releases are available from
the Python 2.5 page, at

http://www.python.org/2.5/highlights.html

Enjoy this release,
Martin

Martin v. Loewis
mar...@v.loewis.de
Python Release Manager
(on behalf of the entire python-dev team)
-- 
http://mail.python.org/mailman/listinfo/python-list

[ANN] Python 2.5.6 released

[Martin v. Löwis]

On behalf of the Python development team and the Python community, I'm
happy to announce the release of Python 2.5.6. There were no changes
since the release candidate.

This is a source-only release that only includes security fixes. The
last full bug-fix release of Python 2.5 was Python 2.5.4. Users are
encouraged to upgrade to the latest release of Python 2.7 (which is
2.7.1 at this point). This release is most likely the final release of
Python 2.5; under the current release policy, no security issues in
Python 2.5 will be fixed after October, 2011.

This releases fixes issues with the urllib, urllib2, SimpleHTTPServer,
and audiop modules. See the release notes at the website (also
available as Misc/NEWS in the source distribution) for details of bugs
fixed.

For more information on Python 2.5.6, including download links for
various platforms, release notes, and known issues, please see:

http://www.python.org/2.5.6

Highlights of the previous major Python releases are available from
the Python 2.5 page, at

http://www.python.org/2.5/highlights.html

Enjoy this release,
Martin

Martin v. Loewis
mar...@v.loewis.de
Python Release Manager
(on behalf of the entire python-dev team)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Small python24.dll / how to strip off asian codecs to separate package(s) ?

[Martin v. Löwis]

Robert wrote:
> Or how to build one?

Just download the source, and follow the instructions in
PCBuild/readme.txt. Then, edit the pythoncore project to remove
the files you don't want to include, and edit config.c to remove
the dangling references.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Small python24.dll / how to strip off asian codecs to separate package(s) ?

[Martin v. Löwis]

Robert wrote:
> Wouldn't it be an issue to think about if future win-python distributions
> should keep on including the asian codecs in the main-dll?

Indeed, it would. As I said before: if somebody defines a clear, fair
policy which finds agreement in the community, I'm willing to change the
current policy on what goes into python24.dll, and what is built
separately. Bonus points if the policy is accompanied with a patch.
The policy should not be static, i.e. it should also give a guideline
for modules introduced in the future.

At the moment, the policy is this: "everything that does not depend on
an external library should go into pythonXY.dll". As a result of this
policy, everybody downloading the sources should be able to build
Python as long as you have the right version of Visual Studio.

The reason for this particular policy is that it simplifies maintenance
if there are only few external .pyd files in the standard distribution.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Memory stats

[Martin v. Löwis]

Tarek Ziadé wrote:
> I am trying to find a general memory profiler that can measure the
> memory usage in Python program
> and gather some stats about object usages, and things like that.

As Diez says, use gc: gc.getobjects() gives you all container objects.
If you want a list of all objects (container or not), you have to
compile a debug build of Python.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Fixes since 2.4.2c1?

[Martin v. Löwis]

Michael Ströder wrote:
> Does that differ from 2.4.2c1? On Monday I noticed a crash in the test
> suite on a box running Solaris 8. It seems I can build Python 2.4.1 and
> run make test there without problems.

There is also a chance that you found a compiler bug. So reporting the
compiler you used would be essential.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RELEASED Python 2.4.2 (final)

[Martin v. Löwis]

Trent Mick wrote:
> It is possible that the python.org installer didn't overwrite the
> "python24.dll" in the system directory (C:\WINDOWS\system32). Try doing
> this:

Even though this is apparently what happened, I'm puzzled as to why it
happened: shouldn't the version number of python24.dll in the pydotorg
installer be higher than the one in the ActivePython installer, and
shouldn't then Windows Installer overwrite the DLL?

The version in the pydotorg installer is 2.4.2150.1012.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RELEASED Python 2.4.2 (final)

[Martin v. Löwis]

Trent Mick wrote:
> I suppose that is possible.  Martin, does python-2.4.2.msi install
> python24.dll to the Python install dir for a "per-user" installation?

Yes, that is supported. It would be good if Bugs could confirm that
he only deleted the wrong python24.dll, with the proper one being
installed in a different place already. His explanation
"you called it, it was an errant python24.dll left over from an old
ActiveState installation." is too imprecise to be able to tell
whether the two python DLLs where in the same location (requiring
a repair step) or in different locations (likely then resulting from
one being per-machine, and the other per-user).

Bugs?

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What encoding is used when initializing sys.argv?

[Martin v. Löwis]

Petr Prikryl wrote:
> I know about the rejected attempt to implement
> sys.argvu. Still, how the sys.argv is filled? What
> encoding is used when parsing the cmd line internally?
> To what encoding is it converted when non ASCII
> characters appear?

Python does not perform any conversion whatsoever.
It has a traditional main() function, with the
char *argv[] argument.

So if you think that the arguments are inherently
Unicode on your system, your question should be
"how does my operating system convert the arguments"?

That, of course, depends on your operating system.
"MS Windows environment" is not precise enough, since
it also depends on the specific incarnation of that
environment. On Windows 9x, I believe the command
line arguments are "inherently" *not* in Unicode,
but in a char array. On Windows NT+, they are Unicode,
and Windows (or is it the MS VC runtime?) converts them
to characters using the CP_ACP code page.

Kind regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: RELEASED Python 2.4.2 (final)

[Martin v. Löwis]

Bugs wrote:
> - I had an old ActiveState python24.dll in \windows\system32 which I
> deleted.  That must've been left over from a 2.4.1 ActiveState
> installation which I had installed then uninstalled sometime ago.
> 
> - I then uninstalled Python 2.4.2 and re-installed it for "all" users.
> After that everything seemed to work properly.

That is somewhat mysterious...

> I'm not sure where the Python 2.4.2 installation put it's python24.dll?

It depends on whether this is a "for all" installation, or a "just for
me" installation. In the "for all" installation, it goes to
\windows\system32. In the "just for me" installation, it goes to
c:\python24 (ie. the directory where python.exe is). The rationale
is that a non-admin user might not have enough permissions to
write to system32.

> Please let me know if you need additional details.

It would be good if you could recall what mode the original 2.4.2
installation had. If it was "just for me", then you might have
installed the 2.4.2 version of python24.dll into the python directory.
Still, it is then puzzling why it would have used the version in
system32, as the application's (i.e. python.exe) directory should
be searched first.

For the record, please also report what operating system you
were using.

The other theory is that the 2.4.2 installer failed to overwrite
the ActivePython version. This should not have happened, though,
since the installer should have noticed that the 2.4.2 version
is newer than the 2.4.1 one (and indeed, in a test installation,
it did so correctly).

It is probably too late to recreate all details, so we should
just watch whether it happens again.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Unicode charmap decoders slow

[Martin v. Löwis]

Tony Nelson wrote:
> Is there a faster way to decode from charmaps to utf-8 than unicode()?

You could try the iconv codec, if your system supports iconv:

http://cvs.sourceforge.net/viewcvs.py/python-codecs/practicecodecs/iconv/

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Unicode charmap decoders slow

[Martin v. Löwis]

Tony Nelson wrote:
> I had seen iconv.  Even if my system supports it and it is faster than 
> Python's charmap decoder, it might not be available on other systems.  
> Requiring something unusual in order to do a trivial LUT task isn't an 
> acceptable solution.  If I write a charmap decoder as an extension 
> module in Pyrex I can include it with the program.  I would prefer a 
> solution that doesn't even need that, preferably in pure Python.  Since 
> Python does all the hard wark so fast it certainly could do it, and it 
> can almost do it with "".translate().

Well, did you try a pure-Python version yourself?

table = [chr(i).decode("mac-roman","replace") for i in range(256)]

def decode_mac_roman(s):
 result = [table[ord(c)] for c in s]
 return u"".join(result)

How much faster than the standard codec is that?

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Call C functions from Python

[Martin v. Löwis]

Java and Swing wrote:
> Is there some other way, besides SWIG, which will allow me to call
> functions inside an Ansi C DLL?

You could write an extension module. See Modules/xxmodule.c in
the Python source tree, as well as

http://docs.python.org/ext/ext.html

In the specific case, if it weren't for the digits argument,
the wrapper function would read

static PyObject*
Py_DoSomeStuff(PyObject*unused, PyObject* args)
{
   char *input;
   if (!PyArg_ParseTuple("s:DoSomeStuff", &input))
 return NULL;
   return PyString_FromString(DoSomeStuff(input));
}

I cannot extend this to digits, as I don't know how
the digits are represented if there is more than one
(i.e. how does DoSomeStuff know how many digits are
being passed?)

If DoSomeStuff returns NULL on errors, additional
exception throwing is necessary. If DoSomeStuff returns
memory that the caller needs to release, you need to
do so before returning from Py_DoSomeStuff.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Replacing utf-8 characters

[Martin v. Löwis]

Mike wrote:
> So it seems link.replace() function reads whether the first option is 
> utf-8 and converts the second option automatically to utf-8?  How do I 
> prevent that?

Not sure what an option is... if you are talking about parameters,
rest assured that .replace does not know or care whether any
of its parameters is encoded in UTF-8. Also not sure where you got
the impression UTF-8 could have to do anything with this.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Compile as static

[Martin v. Löwis]

[EMAIL PROTECTED] wrote:
> However the executable size is always the same :/ Please assist.

At a minimum, you should edit Modules/Setup to compile all extension
modules in.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PyObject_New

[Martin v. Löwis]

Jeremy Moles wrote:
>   PyObject* obj = _PyObject_New(&PyType_MyType);
>   obj = PyObject_Init(obj, &PyType_MyType);
>   
>   ...
> 
>   return obj;

The call to PyObject_Init is redundant: _PyObject_New
is malloc+init. However, this shouldn't cause any crashes (except in the
debug build). PyObject_Init is documented as

Initialize a newly-allocated object op with its type and initial 
reference. Returns the initialized object. If type  indicates that the 
object participates in the cyclic garbage detector, it is added to the 
detector's set of observed objects. Other fields of the object are not 
affected.

[I don't know where the mentioning of GC comes from - it appears to be
  incorrect]

> When "obj" gets back to the interpreter, Python sees it (or rather, it's
> __repr__) in accordance with what it "should" be. However, any attempt
> to USE the object results in a segfault. I feel my problem is simply
> that I'm not allocating "obj" correctly in the C++ function.

It doesn't crash because of the allocation - this code is correct.
However, it is also incomplete: none of the state of the new object
gets initialized in the fragment you are showing. So it likely crashes
because the members of the object are stray pointers or some such,
and accessing them causes a crash.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [regex] case-splitting strings in unicode

[Martin v. Löwis]

John Perks and Sarah Mount wrote:
> I have to split some identifiers that are casedLikeThis into their
> component words. In this instance I can safely use [A-Z] to represent
> uppercase, but what pattern should I use if I wanted it to work more
> generally? I can envisage walking the string testing the
> unicodedata.category of each char, but is there a regex'y way to denote
> "uppercase"?

In this form, it is currently not implemented, although it should be
(written as [[:upper:]], I believe); contributions are welcome (make
sure you read the Unicode consortium's guidelines on regular expressions
before attempting to implement it).

Until then, the "best" way is to use a regular character class,
precomputed or computed at runtime.

uni_upper = [unichr(i) for i in range(sys.maxunicode) if 
unichr(i).isupper()]
uni_re = u"["+u"".join(uni_upper)+u"]"

On my machine, this takes approximately one second to compute,
which may or may not be too much as a startup cost. To speed
this up, you could dump the resulting uni_re into a Python
source file.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Looking for info on Python's memory allocation

[Martin v. Löwis]

Fredrik Lundh wrote:
I = iter(L)
I
> 
> 
> 
len(I)
> 
> 3
[...]
> (it's probably not a good idea to rely on this behaviour...)

I believe this has been classified as a bug in Python 2.4, which
will be undone in Python 2.5.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python name lookups / Interning strings

[Martin v. Löwis]

Dave wrote:
> What exactly does it mean to "intern" a string?

"exactly" it means to place lookup the string in the
global interning dictionary. If an entry is found,
then interning yields the string in the interning dictionary.
If the string is not found, it is added to the interning
dictionary, and returned; its "interning state" is set
to "interned" (either mortally or immortally, depending
on the exact interning request).

More colloquially, it means to find a unique representative
of the set of all equal-valued strings (e.g. one unique
representative for all strings equalling "close"). If
a and b are variables referring to interned strings,
'a==b' implies 'a is b' (whereas usually only the
reverse is true).

HTH,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why asci-only symbols?

[Martin v. Löwis]

Mike Meyer wrote:
> Out of random curiosity, is there a PEP/thread/? that explains why
> Python symbols are restricted to 7-bit ascii?

No PEP yet; I meant to write one for several years now.

The principles would be
- sources must use encoding declarations
- valid identifiers would follow the Unicode consortium guidelines,
   in particular: identifiers would be normalized in NFKC (I think),
   adjusted in the ASCII range for backward compatibility (i.e.
   not introducing any additional ASCII characters as legal identifier
   characters)
- __dict__ will contain Unicode keys
- all objects should support Unicode getattr/setattr (potentially
   raising AttributeError, of course)
- open issue: what to do on the C API (perhaps nothing, perhaps
   allowing UTF-8)

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: C Extension - return an array of longs or pointer?

[Martin v. Löwis]

Brandon K wrote:
> PyTuple_SET_ITEM(py, i, Py_BuildValue("l",*result[i]);

Using Py_BuildValue is overkill here. PyInt_FromLong will do just
as well.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: installer for amd64 build of python?

[Martin v. Löwis]

[EMAIL PROTECTED] wrote:
> Does anyone know of an available downloader for an amd64 bit
> build of (a modern) python?

There is no official build. The only official Win64 build is
for Itanium, see www.python.org/2.4.2

> I've done my due diligence search of python docs and
> mailing archives, and I'm somewhat mystified by the lack of info
> on this topic.

I'm not at all mystified. The operating system was only released
after the release of Python 2.4, there is no official compiler
available yet to the general public, and I don't have the hardware
to test such a release.

It's fairly likely that Python 2.5 will not see official AMD64
Windows binaries, either.

> I'm embedding python in a native 64 bit
> application so python32 using WOW64
> is not an option for me.

So you should build this on your own.

> Alternatively, does anyone have experience
> of using the visual studio files in the PCBuild
> directory under amd64?

A number of people have tried this. There is one open issue
with signal handling: VS 2005 will apparently drop compatibility
with Standard C in one aspect where previous releases used to
be C compatible (namely, processing of unsupported signal
numbers in signal(3)). Because of that, Python will raise
an assertion failure in the debug build.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why asci-only symbols?

[Martin v. Löwis]

Bengt Richter wrote:
> Perhaps string equivalence in keys will be treated like numeric equivalence?
> I.e., a key/name representation is established by the initial key/name 
> binding, but
> values can be retrieved by "equivalent" key/names with different 
> representations
> like unicode vs ascii or latin-1 etc.?

That would require that you know the encoding of a byte string; this
information is not available at run-time.

You could also try all possible encodings to see whether the strings
are equal if you chose the right encoding for each one. This would
be both expensive and unlike numeric equivalence: in numeric 
equivalence, you don't give a sequence of bytes all possible
interpretations to find some interpretation in which they are
equivalent, either.

There is one special case, though: when comparing a byte string
and a Unicode string, the system default encoding (i.e. ASCII)
is assumed. This only really works if the default encoding
really *is* ASCII. Otherwise, equal strings might not hash
equal, in which case you wouldn't find them properly in a
dictionary.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why asci-only symbols?

[Martin v. Löwis]

Bengt Richter wrote:
> Well, what will be assumed about name after the lines
> 
> #-*- coding: latin1 -*-
> name = 'Martin Löwis' 
> 
> ?

Are you asking what is assumed about the identifier 'name', or the value
bound to that identifier? Currently, the identifier must be encoded in 
latin1 in this source code, and it must only consist of letters, digits,
and the underscore.

The value of name will be a string consisting of the bytes
4d 61 72 74 69 6e 20 4c f6 77 69 73

> I know type(name) will be  and in itself contain no encoding 
> information now,
> but why shouldn't the default assumption for literal-generated strings be 
> what the coding
> cookie specified?

That certainly is the assumption: string literals must be in the
encoding specified in the source encoding, in the source code file
on disk. If they aren't (and cannot be interpreted that way), you
get a syntax error.

> I know the current implementation doesn't keep track of the different
> encodings that could reasonably be inferred from the source of the strings, 
 > but we are talking about future stuff here ;-)

Ah, so you want the source encoding to be preserved, say as an attribute
of the string literal. This has been discussed many times, and was
always rejected.

Some people reject it because it is overkill: if you want reliable,
stable representation of characters, you should use Unicode strings.

Others reject it because of semantic difficulties: how would such
strings behave under concatenation, if the encodings are different?

> #-*- coding: latin1 -*-
> name = 'Martin Löwis' 
> 
> could be that name.encoding == 'latin-1'

That is not at all intuitive. I would have expected name.encoding
to be 'latin1'.

> Functions that generate strings, such as chr(), could be assumed to create
> a string with the same encoding as the source code for the chr(...) 
> invocation.

What is the source of the chr invocation? If I do chr(param), should I 
use the source where param was computed, or the source where the call
to chr occurs? If the latter, how should the interpreter preserve the
encoding of where the call came from?

What about the many other sources of byte strings (like strings read 
from a file, or received via a socket)?

> This is not a fully developed idea, and there has been discussion on the 
> topic before
> (even between us ;-) but I thought another round might bring out your current 
> thinking
> on it ;-)

My thinking still is the same. It cannot really work, and it wouldn't do 
any good with what little it could do. Just use Unicode strings.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why asci-only symbols?

[Martin v. Löwis]

Bengt Richter wrote:
>>Others reject it because of semantic difficulties: how would such
>>strings behave under concatenation, if the encodings are different?
> 
> I mentioned that in parts you snipped (2nd half here):
> 
> This could also support s1+s2 to mean generate a concatenated string
> that has the same encoding attribute if s1.encoding==s2.encoding and 
> otherwise promotes
> each to the platform standard unicode encoding and concatenates those if they
> are different (and records the unicode encoding chosen in the result's 
> encoding
> attribute).

It remains semantically difficult. There are other alternatives, e.g.
(s1+s2).encoding could become None, instead of using your procedure.

Also, this specification is incomplete: what if either s1.encoding
or s2.encoding is None?

Then, what recoding to the platform encoding fails? With ASCII
being the default encoding at the moment, it is very likely that
concatenations will fail if there are funny characters in either
string.

If you propose that this should raise an exception, it means that
normal string concatenations will then give you exceptions just
as often as (or even more often than) you get UnicodeErrors
currently. I doubt users would like that.

>>>This is not a fully developed idea, and there has been discussion on the 
>>>topic before
>>>(even between us ;-) but I thought another round might bring out your 
>>>current thinking
>>>on it ;-)
>>
>>My thinking still is the same. It cannot really work, and it wouldn't do 
>>any good with what little it could do. Just use Unicode strings.
>>
> 
> To hear "It cannot really work" causes me agitation, even if I know it's not 
> worth
> the effort to pursue it ;-)

It is certainly implementable, yes. But it will then break a lot of 
existing code.

> Though I grant you
> 
> #-*- coding: latin1 -*-
> name = u'Martin Löwis' 
> print name
> 
> is not that hard to do.

This is indeed what you should do. In Python 3, you can omit the u,
as the string type will go away (and be replaced with the Unicode type).

> (Please excuse the use of your name, which has a handy non-ascii letter ;-)

No problem with that :-)

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Extention Module/ list of chars-> string

[Martin v. Löwis]

Tuvas wrote:
> A. Change a list of chars to a single string or
> B. Read a list of chars in an extention module

Either is possible, but I recommend to do A:

data = ''.join(data)

Then pass the modified data to the extension module.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to statically link Python with ncurses and readline?

[Martin v. Löwis]

[EMAIL PROTECTED] wrote:
> That is, there are no longer any readline.so or _curses.so in lib-dynload, 
 > but instead they are added to Python's shared library dependencies.

What is "they" in that sentence? readline.so and _curses.so? I very much
doubt that - more likely, libreadline.so and libncurses.so are in
the dependencies

 > Any clue as to what's going on here would be much appreciated.

Instead of linking with -lncurses, link explicitly with
/usr/lib/libncurses.a; or else using -static as a linker flag.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to statically link Python with ncurses and readline?

[Martin v. Löwis]

[EMAIL PROTECTED] wrote:
> Well, I implied that _curses.so and readline.so disappeared as one
> would expect, dependencies on libncurses.so and libreadline.so instead
> showing up in Python itself. It's very strange that this happens with
> ncurses and readline, when both db and Tcl/Tk got linked in with no
> fuss. Anyway, I built static-only (.a) versions of readline and
> ncurses, which did the trick.

Ah, you didn't *have* static versions of readline and ncurses...
So how should it possibly have linked those libraries statically???

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to statically link Python with ncurses and readline?

[Martin v. Löwis]

[EMAIL PROTECTED] wrote:
> What do you mean? A static-only build does somehow exclude that I had
> static libraries before?

No, it doesn't. I misunderstood. It is quite unclear still what you had
been doing: e.g. did you have shared versions of db and Tcl or not?
Did you have linker scripts in place? What was your specific command
line you have used to link the entire interpreter?

In the absence of details, we have to take guesses on Usenet. Sometimes,
the guesses are right, and sometimes they are wrong.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Building 2.4.2 on OpenBSD 3.8

[Martin v. Löwis]

Dan M wrote:
> /usr/include/sys/event.h:53: error: syntax error before "u_int"
> /usr/include/sys/event.h:55: error: syntax error before "u_short"
> *** Error code 1
> 
> Stop in /usr/local/src/Python-2.4.2.
> 
> immediately upon running "make". I took at look at event.h to see if I
> could find and fix the problem, but the error isn't immediately apparent
> to me.

In that case, it would help if you would report what line 53 of event.h
precisely is. Typically, if it says "syntax error before", it means
"immediately, right away before", or "the error actually is *in*
u_int" (the latter indicating that u_int is not defined at that point).

Generating a preprocessor output then helps, seeing whether something
got #define'd in some strange sense, or whether u_int was defined before
use.

Most likely, it is the old BSD bug of not supporting POSIX correctly.
Somebody should really work with the system developers to get this
resolved; it's been several years now that the bug is present.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: urlencode with high characters

[Martin v. Löwis]

Jim wrote:
> My understanding is that I am supposed to be able to urlencode anything
> up to the top half of latin-1 -- decimal 128-255.

I believe your understanding is incorrect. Without being able to quote
RFCs precisely, I think your understanding should be this:

- the URL literal syntax only allows for ASCII characters
- bytes with no meaning in ASCII can be quoted through %hh in URLs
- the precise meaning of such bytes in the URL is defined in the
   URL scheme, and may vary from URL scheme to URL scheme
- the http scheme does not specify any interpretation of the bytes,
   but apparantly assumes that they denote characters, and follow
   some encoding - which encoding is something that the web server
   defines, when mapping URLs to resources.

If you get the impression that this is underspecified: your impression
is correct; it is underspecified indeed.

There is a recent attempt to tighten the specification through IRIs.
The IRI RFC defines a mapping between IRIs and URIs, and it uses
UTF-8 as the encoding, not latin-1.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: /usr/lib/python2.4/posixfile.py error

[Martin v. Löwis]

Chris wrote:
> Wonder if anyone can help me?

I very much doubt that, with as little information you gave us.

> I am trying to run a perl script but I keep getting this error:
> 
> /usr/lib/python2.4/posixfile.py:59: DeprecationWarning: The posixfile 
> module is obsolete and will disappear in the future
>   DeprecationWarning)

It's very strange that you get a Python error when running a Perl
script.

> Any ideas on why it's failing on that python module?

It's not failing. It's a warning, not an error. The warning is
that the Python code you are running uses the posixfile module,
and it should be rewritten to stop doing so, as the posixfile
module will go away eventually.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PYTHON LOOSING FOR JAVA???????

[Martin v. Löwis]

Fcamattti wrote:
> Hello for everybody
> 
>  So I have a doubt. I'd like to know what do you think about the joint
> of efforts of Sun Microsystems and the Google to create a office web
> based. I sincerely enjoy the idea althoug I'd like to know what will be
> the future of this wonderful language called Python??

I think I know Although, the future is difficult to predict???
Maybe people will not care what Sun Microsystems and Google tell
them???

¡Kind regards!
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: weird problem with os.chmod

[Martin v. Löwis]

James Colannino wrote:
> I discovered the solution and thought I'd share it with everyone.  I 
> discovered as I googled that Python used to have a function called 
> atoi() that took the parameter base=X.  I decided to see if that worked 
> with the newer function int() and it did :)

I recommend that you get used to reading documentation,
instead of merely googling. help(int) would have revealed that.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: weird problem with os.chmod

[Martin v. Löwis]

James Colannino wrote:
> Ok, so further research revealed that 0600 is actually the octal 
> representation for 384 (which makes sense.)  So then, I guess my 
> question would have to be, is there a way for me to make Python aware 
> that the 0600 I'm passing to int() is octal and not decimal so that I 
> will get 384 instead of 600?

Try this:

http://docs.python.org/lib/built-in-funcs.html#l2h-39

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Want to perform unattended installation of SW using python

[Martin v. Löwis]

28tommy wrote:
> I'm trying to automate an installation of some SW that is installed on
> Windows (you know - 'Start' ==> 'Next' ==> 'Next' ==> 'Finish' kind of
> installation). Is ther any way to perform that using python?

While that is possible in Python, it is very tedious to do: you 
essentially have to generate mouse clicks for the application, using
the Win32 API.

Most packaging tools provide some support for unattended installation,
so I would rather research what options to pass to the installer to
make it non-interactive.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tix BUG? Where to submit?

[Martin v. Löwis]

Ron wrote:
> Surprising to myself, this was all that it took to make this work.  So I'm 
> not sure why this method was left out.  Could it have been an oversight?

It could be an oversight. More likely, it was not originally there when 
the underlying tix library was developed, and only added to tix later;
then, it was never added to Tix.py.

> Anyway, where would I subit this report to have it considered that this be 
> added to Tix?

As Jeff explains: sf.net/projects/python, preferably as a patch (unified
diff).

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: compiling Python under Windows

[Martin v. Löwis]

Philippe C. Martin wrote:
>>My mistake: The makefile (as written in the readme!) looks for bzip 1.0.2
>>
> 
> PS: since bzip.org does not have 1.0.2 source anymore, can I just rename
> 1.0.3 ?

That should work; alternatively, you can change the project file.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to install python2.4.2 on IRIX6.5 ?

[Martin v. Löwis]

Xiao Jianfeng wrote:
> self.assertEquals(result, expected)
> AssertionError: '-0' != '0'
> 
> Can somebody tell me what's the problem ?

It looks like there is some minor bug in the floating
point libraries of your operating system: Python expects
that the strings for both numbers are the same in the test,
but actually, one number is formatted as -0, and the other
as 0.

It's not clear what the test_locale failure is; when
you run it again, it says that test_frozen has been run
instead. Are you sure the commands worked precisely as
entered? In any case, test_locale failures are typically
also of minor importance only, as well.

This is a minor issue only, and shouldn't affect Python's
functionality for most cases. Given that the rest of
the test suite passed, just go ahead and make install.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: bsddb185 question

[Martin v. Löwis]

thakadu wrote:
> So it seems I am forced to use the bsddb185 module
> which does not have convenient record level methods.
> Am I missing something here or have others eperienced
> tha same?

I think you are missing that bsddb185 implements the
dictionary interface. So you can use [key] to access
the value for a certain key.

Regards,
Martin

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: IPv6 RFC 3542 with socket module?

[Martin v. Löwis]

Roy Smith wrote:
> Is there any way to access the RFC 3542 defined macros such as 
> ICMP6_FILTER_SETPASSALL() from within the socket module? 

No. Not sure it would belong into the socket module, though:
an icmp module could be developed independently from the socket
module if needed.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: linking one extension module to another (Mac OSX)

[Martin v. Löwis]

Simon Burton wrote:
> I'm having some trouble linking one extension module to another because
> the linker expects a "lib" prefix and my python modules cannot have
> this prefix.

This is a Good Thing (tm) :-) Don't link extension modules to each
other; this is really asking for trouble. Instead, come up with a
function pointer API in one module, put that into a CObject, and
access the CObject through import statements.

Alternatively, make both extension modules link to the same
backend library.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: keys in dictionary

[Martin v. Löwis]

Shi Mu wrote:
> I run the following code and got wrong message, but I still want to 
> make [1,2],[4,3] and [6,9] to be keys of the dictionary or change the
> style a little bit. How to do that?

Make them tuples:

 >>> p=[[1,2],[4,3],[6,9]]
 >>> n=dict([(tuple(x),[]) for x in p])
 >>> n
{(6, 9): [], (1, 2): [], (4, 3): []}

Regards,
Martin

P.S. Please don't cross-post.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: bsddb185 question

[Martin v. Löwis]

thakadu wrote:
> It seems it doesnt implement ALL of the dictionary interface though.
> dir({}) yields many more methods than dir(bsddb185.open(f)).
> So bsddb185 is missing many of the methods that I am used
> to in bsddb. I mentioned some above that are missing, pop()
> in particular would be useful in my situation but there are
> others as well.

I see. But .pop can be easily emulated using getitem, then del:

def bsddb_pop(db, key):
 result = db[key]
 del db[key]
 return result

>>From a java perspective I believe that its not possible to omit certain
> methods when implementing an interface, I am not sure about Python
> though.

In Python, I'm sure: it is possible.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: linking one extension module to another (Mac OSX)

[Martin v. Löwis]

[EMAIL PROTECTED] wrote:
> I have C Extension classes distributed across several modules with
> non-trivial interdependancies. I guess you are saying I should have
> these in backend libraries and then put the module specific functions
> in the module itself. It's going to be tricky because I am using
> distutils and pyrex to do all this. Maybe Greg (Ewing) has some other
> ideas.

Alternatively, if you are always shipping the entire set: make
them all a single extension module.

For backwards compatibility, provide Python modules with the "old"
module names, which export the symbols that used to be in that
module.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Built windows installers and Cygwin

[Martin v. Löwis]

SPE - Stani's Python Editor wrote:
> Is there any way for these kind of installers to be used with cygwin.

No. The bdist_wininst packages link against Microsoft's C library;
the cygwin Python against Cygwin's.

If there is no native code in the package, it would be possible to
*use* them - installing them still won't work. To use them, unzip
them, and spread the files as you see fit.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using gettext to provide different language-version of a script

[Martin v. Löwis]

Thomas W wrote:
> Hmmm ... any hints?

I think gettext needs environment variables (e.g. LANG)
to be set.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: bsddb185 question

[Martin v. Löwis]

thakadu wrote:
> However the line:
> del db[key]
> results in an error: (1, 'Operation not permitted')
> (only tested on Python 2.3.5)

Did you open the dbm file for read-write? This should work,
and is implemented.

> Could this be because the .del() method of the dictionary
> has not been implemented either? In fact in my tests
> any attempt at altering the db by use of normal dictionary
> methods fails with the same error. e.g.
> db['newkey']='newvalue' causes the same error.

Ah, so you definitely should pass the "w" argument to the
open function.

> Thanks for your info on the rules for interface implementation
> in Python. That is an interesting difference. I have grown to
> like the rules for java interface implementation because you
> are guaranteed that all defined methods for the interface have
> been implemented. That is what is meant by "implementing" an
> interface in java.

In Python, there is also the notion of implementing an interface.
However, the interface is just specified in English text,
and the interpreter never checks whether all of them are
implemented. For mapping types,

http://www.python.org/doc/current/lib/typesmapping.html

gives the list of operations they ought to support.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: 2.4.2 on AIX fails compiling _codecs_cn.c

[Martin v. Löwis]

Paul Watson wrote:
> Any ideas why ./Modules/cjkcodecs/_codecs_cn.c fails to compile?  It 
> appears that the CODEC_STATELESS macro is concatenating 'hz' with a 
> number and text.

More likely, hz is already defined to be 100, then forming 100_encode.

It would be best if you could find out what AIX header file defines
hz to be 100, and whether there is any way to supress that definition.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: 2.4.2 on AIX fails compiling _codecs_cn.c

[Martin v. Löwis]

Paul Watson wrote:
> This is on AIX 4.3.3
> 
> $ grep -i _hz $(find . -name m_param.h)
> #define _HZ 100/* ticks per second of the clock*/
> #define __hzHZ /* Berkeley uses lower case hz  */
> #define HZ  _HZ
> #define hz  __hz

I expected to see something like this. However: in the file containing
the #define hz: Is there any #ifdef around it that could be 
disabled/enabled?

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: 2.4.2 on AIX fails compiling _codecs_cn.c

[Martin v. Löwis]

Paul Watson wrote:
> Can we #undef _ALL_SOURCE for _codecs_cn.c compilation?

Where does _ALL_SOURCE come from? Why is it defined?
What is its effect on hz?

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Embedding a CPython Script engine in a .NET application.

[Martin v. Löwis]

Carl Waldbieser wrote:
> Has anyone had any experience embedding a CPython engine in a .NET
> application?  In the COM/ActiveX world, it was pretty easy to use Mark
> Hammond's win32 modules to create a script engine component that you could
> expose other COM objects to, but I was not sure how I would go about doing
> something similar in a .NET environment.

I don't have experience with this specific problem, but in general,
.NET is really good at calling COM objects (in fact, this was
historically the sole reason for creating .NET: to make COM better
usable).

So just import, in VS.NET, the Python interpreter to your project
(Add Reference), and use the C# functions that the COM interop
code gives you.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is dictionary.keys() a list and not a set?

[Martin v. Löwis]

Christoph Zwerschke wrote:
> Anyway, I was thinking about whether it would be possible and desirable 
> to change the old behavior in future Python versions and let dict.keys() 
> and dict.values() both return sets instead of lists.

You answer the question whether it would be possible (no, because of
backwards compatibility); you don't attempt to answer the question
whether it would be desirable. Why do you think that would be
a good idea?

If you want the set of keys of a dictionary d, then do set(d):

 >>> d={1:2,3:4}
 >>> set(d)
set([1, 3])

As Mike Meyer explains, the same is meaningless for .values():
they might not be unique. For .items(), conversion into a set
does would appear to be meaningful, but doesn't actually work:
if the values contain unhashable objects, the set of tuples
cannot be constructed (as set members must be immutable).

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: 2.4.2 on AIX fails compiling _codecs_cn.c

[Martin v. Löwis]

Paul Watson wrote:
> It appears that _ALL_SOURCE gets defined in the /usr/include/standards.h 
> file.  If we could #define _ANSI_C_SOURCE or _POSIX_SOURCE, it appears 
> that it would eleminate _ALL_SOURCE.

Ah, ok - this should be easy enough. Python would normally define
_POSIX_SOURCE (through _XOPEN_SOURCE), but configure(.in) has this
block:

   # On AIX 4 and 5.1, mbstate_t is defined only when _XOPEN_SOURCE == 
500 but
   # used in wcsnrtombs() and mbsnrtowcs() even if _XOPEN_SOURCE is not 
defined
   # or has another value. By not (re)defining it, the defaults come in 
place.
   AIX/4)
 define_xopen_source=no;;
   AIX/5)
 if test `uname -r` -eq 1; then
   define_xopen_source=no
 fi
 ;;

which causes _XOPEN_SOURCE (and subsequently probably _POSIX_SOURCE) not 
to be defined. What AIX version are you using? Can you try removing
the fragment from configure(.in), rerun configure, verify that 
_XOPEN_SOURCE is defined in pyconfig.h, and then try building again?

If this works, this might be a solution. Otherwise, we need to put 
something like this into _codecs_cn.c:

#ifdef hz
/* On AIX version such-and-such, hz is defined because _ALL_SOURCE is
defined, this in turn is defined because _XOPEN_SOURCE is not.
As _XOPEN_SOURCE cannot be enabled (see configure.in), we just
work around by removing the hz definition again. */
#undef hz
#endif

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: 2.4.2 on AIX fails compiling _codecs_cn.c

[Martin v. Löwis]

Paul Watson wrote:
> I also note that compiles occurring after the complaint about not 
> finding Tcl/Tk do not appear to get the OPT= setting I specified on the 
> 'make' command line.  It starts with compilation of structmodule.c and 
> includes the _codecs_??.c files.  Does this have any significance?  Is 
> it possible that there are other settings not being used?

At this point, I'm giving up. I would need access to an AIX and more
time than I have to be of further help.

Good luck,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is dictionary.keys() a list and not a set?

[Martin v. Löwis]

Christoph Zwerschke wrote:
> I know. But this does not answer the question: If the keys *are* already 
> a set according to their semantics, why aren't they returned as a set 
> from the beginning?

Notice that this was not the question you originally asked. This
one has a clear answer: because that was always the case, and because
it was never changed.

Your original question was "could it be changed, and should it be
changed?" As the answer to the first part is already "no", the
second part really doesn't raise.

> This, of course, in turn raises the question ;-) Would it be desirable 
> to have an additional, more general set datatype that can contain 
> mutable objects? I know about the perfomance drawbacks. And I think I 
> know the answer: It would not make much sense, since the implementation 
> would be actually not different from a list. So you could take a list 
> anyway. Which gives the answer why values() and items() return a list.

Actually, this answer is only partial. The real question is a semantical
one:

In a set, all elements are different. In Python, this means that, for
any two elements X and Y of a set, X!=Y. Now, consider this set:

a = [1]
b = [1,2]
S = set(a,b)
a.append(2)

Now, a==b, so the inherent set property breaks. In theory, this should
cause S to contain only a single element; the other one should
disappear. This is not only hard to implement (removal from a set
would have to remove all duplicates, iterating over a set would have
to skip over duplicates) - there is also another semantical issue:
If one element is skipped/dropped, which of these (a or b)?

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is dictionary.keys() a list and not a set?

[Martin v. Löwis]

Christoph Zwerschke wrote:
> Sorry. Your answer was good; I missed the point and thought you wrote 
> set(d.keys()). Is it documented anywhere that set(d) = set(d.keys())? I 
> think this should go into the Python Doco where keys() are explained.

It follows from what is documented. set() creates a
set that contains all elements in the iterable object:

http://docs.python.org/lib/built-in-funcs.html#l2h-63

Now, is a dictionary an iterable object? Yes, it is:

http://www.python.org/peps/pep-0234.html

Together, this gives the property I demonstrated.

Unfortunately, the PEP apparently hasn't made it into the
implementation.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is dictionary.keys() a list and not a set?

[Martin v. Löwis]

Mike Meyer wrote:
>>I trusted the doco which defines a set as "an unordered collection of
>>immutable values" (http://docs.python.org/lib/types-set.html).
> 
> 
> If the hash changes, you've screwed up the set. What it really should
> say is "collection of objects with fixed hashes", or words to that
> effect. Do you want to file a PR on this?

It's more than that: you really mean "hashable values", where "hashable"
does not just imply that the hash value is fixed, but also that the
equal objects hash same.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is dictionary.keys() a list and not a set?

[Martin v. Löwis]

Christoph Zwerschke wrote:
> You need to know a third thing, namely that as an iterable object, a 
> dictionary returns the keys only, not the key/value tuples which would 
> also make some sense. If it had been designed that way, you could write

Right. I forgot to say that, but that is also explained in PEP 234.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is dictionary.keys() a list and not a set?

[Martin v. Löwis]

Mike Meyer wrote:
> I would have phrased it as "identical" instead of equal. Sets should
> hold unique elements, where "unique" depends on the use at hand. For
> some uses, "==" might be appropriate. For others, "is" might be
> appropriate.

Sets, as currently defined, require hashable objects, i.e. objects
which has same all the time and hash same as all other equal objects.
Phrasing it as "identical" would be incorrect: hashes currently operate
on equality, not identity.

> Would you care to suggest an improved wording?

No - I have learned that I cannot write documentation that others can
understand. I usually write documentation which only attempts to be
correct, and others than make it understandable.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Which License Should I Use?

[Martin v. Löwis]

mojosam wrote:
> How do I decide on a license? 

If you want to look at this from a legal point of view,
I highly recommend reading Larry Rosen's book, Open Source
Licensing. It covers issue you would never have thought
of without studying law, and gives good advice (IMHO).

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Move xml.sax.saxutils.*?

[Martin v. Löwis]

[EMAIL PROTECTED] wrote:
> It seems like functions such as xml.sax.saxutils.escape and unescape
> are generally useful, and not at all tied to the xml.sax module.  Would
> it make sense to move them somewhere else, like to xml?

Moving it would be a bad idea. Exposing it elsewhere would be useful.

The precise details are best discussed at [EMAIL PROTECTED] While this
is obviously wrong, it is not at all obvious what the right thing to do
is (it never is when dealing with XML).

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is dictionary.keys() a list and not a set?

[Martin v. Löwis]

Mike Meyer wrote:
>>class mylist1(list):
>> def __hash__(self): return 0815
>>
>>class mylist2(list):
>> def __hash__(self): return id(self)
>>
>>In the case of mylist1, everything is ok including semantics, but
>>performance suffers dramatically. In mylist2, performance is great,
>>but semantics suffers greatly.
>>Which of these user-defined types would you call "hashable"?
> 
> 
> The latter two, as the given __hash__ meets the specifications for
> __hash__.

This is not true. The second definition of __hash__ does not meet
the specifications:

http://docs.python.org/ref/customization.html

"The only required property is that objects which compare equal have the
same hash value"

However, I can have two instances of mylist2 which compare equal,
yet have different hash values.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is dictionary.keys() a list and not a set?

[Martin v. Löwis]

Christoph Zwerschke wrote:
>> This is not true. The second definition of __hash__ does not meet
>> the specifications:
>> "The only required property is that objects which compare equal have the
>> same hash value"
> 
> 
> As Mike has written in his last posting, you could easily "fix" that by 
> tweaking the equality relation as well. So technically speaking, Mike is 
> probably right.

No. If you define both __hash__ and __eq__ consistently, then __hash__
would meet the specification. As posted in the example, __hash__ does
not meet the specification, contrary to Mike's claim that it does.

> It would completely break common-sense semantics, 
> because for mylist2, if I have a=[1] and b=[1] then I will have a!=b. 
> This would not be very reasonable, but probably allowed by Python.

If you have a=[1] and b=[1], then *always* a==b; you cannot
change the semantics of list displays. To achieve !=, you
would need to have a=mylist2([1]), b=mylist([2]). Whether or not
it is common sense that classes named "mylist2" compare equal
if they consist of the same items, I don't know - I personally don't
have any intuition as to what a class named "mylist2" should or
should not do.

Just that it has "list" in its name provides not sufficient clue: for
example, IdentityDictionary has "dictionary" in its name, yet I would
*not* expect that equality is used to compare keys, but instead I
would expect that identity is used.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is dictionary.keys() a list and not a set?

[Martin v. Löwis]

Mike Meyer wrote:
>>This is not true. The second definition of __hash__ does not meet
>>the specifications:
> 
> 
> The specification isn't on the __hash__ method, but on objects.

What does it mean for a specification to be "on" something? The
specification I quoted is listed "under" the __hash__ heading
of the reference manual.

> Instances of mylist2 that aren't otherwise tweaked won't meet the
> specification.

Right, that's what I meant.

> That's pretty flaky. The same docs say:
> 
> Should return a 32-bit integer usable as a hash value for dictionary
> operations.

What is flaky about this?

> the dictionary implementation requires that a key's hash value is
> immutable

Indeed, the question here is: what does "an object is immutable"
mean in this context? You appear to read it as "an object whose
state does not change", however, what it really means is "an object
which will always use the same state in __hash__ and __eq__".

> Maybe those are only "recommended" properties, but any class that
> doesn't meet them is going to have instances with interesting
> behaviors in sets and as dictionary keys.

Indeed, you can anything return you want in __hash__, or always
raise an exception. What precisely the result of dictionary
operations is when you do so is undefined in Python.

> That's a problem with mylist2 instances, and may or may not be a
> problem in the mylist2 __hash__ method. You can't tell just by
> examining the __hash__ method whether or not it meets this
> specification.

Correct. I would expect that this is the case for any specification
of __hash__ you can come up with: __hash__ would typically refer
to the state of the object, and whether or not the results meets
some specified requirement would also depend on what the state is.
(unless either the specification is trivial (e.g. "may or may
not return a value") or the implementation is trivial (e.g.
"return 0"))

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: New docs for set elements/dictionary keys (Was: Why is dictionary.keys() a list and not a set?)

[Martin v. Löwis]

Mike Meyer wrote:
> Personally, I think we'd be better off to come up with a term for this
> property that doesn't have a commonly understood meaning that has such
> broad areas of disagreement with the property. I've been using
> "hashable", which I would currently define as "has a __hash__ method
> with the properties described in the __hash__ documentation, or does
> not have either a __cmp__ or a __eq__ method."

I would like to use "hashable" as a term as well, but it appears that
many people would understand that to mean "has a __hash__ 
implementation" (i.e. hash(x) returns a value, instead of raising an
exception).

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why is dictionary.keys() a list and not a set?

[Martin v. Löwis]

Christoph Zwerschke wrote:
> Anyway, the original question was: Are mylist1 and mylist2 (as above) to 
> be considered "hashable" types or not?

I long forgot was the original question was (I thought it was
"Why is dictionary.keys() a list and not a set?" :-); anyway,
the answer to this question is certainly "yes".

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: New docs for set elements/dictionary keys

[Martin v. Löwis]

Mike Meyer wrote:
>>>Personally, I think we'd be better off to come up with a term for this
>>>property that doesn't have a commonly understood meaning that has such
>>>broad areas of disagreement with the property. I've been using
>>>"hashable", which I would currently define as "has a __hash__ method
>>>with the properties described in the __hash__ documentation, or does
>>>not have either a __cmp__ or a __eq__ method."
>>
>>I would like to use "hashable" as a term as well, but it appears that
>>many people would understand that to mean "has a __hash__
>>implementation" (i.e. hash(x) returns a value, instead of raising an
>>exception).
> 
> 
> Well, the two aren't quite the same thing: hash returns a value on
> some things that don't have a __hash__. But that is close to the
> meaning I want to give it. What meaning did you have in mind?

Me, personally, I had your definition in mind: hashable should indicate
"returns a value constant over time and consistent with comparison".

I suggested that most people would consider "hashable" to mean:
hash() returns a value. To those people, it is a minor detail whether 
you get the fallback implementation of hash() or whether there is a
default __hash__ implementation for all objects that don't otherwise
define __hash__.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: New Ordered Dictionery to Criticise

[Martin v. Löwis]

Fuzzyman wrote:
> Criticism solicited (honestly) :-)

A couple of minor points:
- I would drop 2.2 compatibility
- self=self isn't needed in the functions, because of
   nested scopes
- popitem(self) can be rewritten as

   def popitem(self):
   try:
   key = self._sequence.pop()
   except IndexError:
   # always try to give the same exception string as
   # dict
   raise KeyError("popitem(): dictionary is empty")
   return key, self.pop(key)

- One problem with the FancyDict is that it allows
   d.keys.append(100)

Regards,
Martin

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: cheese shop registration error

[Martin v. Löwis]

Todd Greenwood-Geer wrote:
> Q: Do I have to be a member of some sort of secret society in order to
> be listed in the 'local recipient table'?

Ah, you don't know about the PSU :-?

Seriously, please report a bug at sf.net/projects/pypi. Unfortunately,
the (single) cheeseshop maintainer is overworked, so that may take
some time.

Regards,
Martin


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: New Ordered Dictionery to Criticise

[Martin v. Löwis]

Fuzzyman wrote:
>>- One problem with the FancyDict is that it allows
>>   d.keys.append(100)
> 
> 
> Are you sure ?

Not at all. This was from inspection only; I propably
misinterpreted the code.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python & Unicode decimal interpretation

[Martin v. Löwis]

Scott David Daniels wrote:
> In reading over the source for CPython's PyUnicode_EncodeDecimal,
> I see a dance to handle characters which are neither dec-equiv nor
> in Latin-1.  Does anyone know about the intent of such a conversion?

To support this:

 >>> int(u"\N{DEVANAGARI DIGIT SEVEN}")
7


> As far as I can tell, error handling is one of:
> strict, replace, ignore, xmlcharrefreplace, or something_else
> What I don't understand is whether, in the ignore or something_else
> cases, there is any chance that digits will show up anywhere that
> they would not if these characters were treated as a character like '?'?
> 
> Can someone either give me definitive "why not" or (preferably) give
> me a test case that shows where that interpretation does not hold.

In the "ignore" case, no output is produced at all, for the unencodable
character; this is the same way that '?' would be treated (it is
also unencodable).

In the something_else case, a user-defined exception handler could
treat the error in any way it liked, e.g. encoding all letters
u'A' to digit '0'. This might be different from the way this error
handler would treat '?'.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: os.rename copies when old is in-use - is this deliberate?

[Martin v. Löwis]

Tony Meyer wrote:
> Is this the intended behaviour?

Sort-of. os.rename invokes the C library's rename, and does whatever
this does. It is expected that most platform's C libraries do what
the documentation says rename does, but platforms may vary in their
implementation of the C library, and from one compiler version to
the other.

Microsoft's documentation of rename/_rename/_wrename is here:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vclib/html/_crt_rename.2c_._wrename.asp

It doesn't say anything about renaming in-use files, either.
Looking at the implementation of _rename, I see that it directly
calls MoveFile, which is documented here:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/movefile.asp

That doesn't mention the in-use case, either.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: os.rename copies when old is in-use - is this deliberate?

[Martin v. Löwis]

Tony Meyer wrote:
> Thanks for that.  In your opinion, would a documentation patch that  
> explained that this would occur on Windows (after the existing note  
> about the Windows rename not being atomic) be acceptable?

In principle, yes. We cannot do that for every platform, of course,
but it probably won't be necessary for any platform, either.

> (The Windows platform C library for Python 2.4+ is in msvcrt71.dll,  
> right?  Does that mean that behaviour will be consistent across  Windows 
> versions, or could 9x/NT/XP/etc all behave differently?)

The behaviour could be different in principle, yes. I found a KB article
that says that MoveFile, on Windows CE 4.0, on a FAT file system, would
move the file even if it was open.

I'm actually also surprised by the behaviour - I would have expected
that the failure to delete the file after the copy would cause an
error, or that the attempt to open the file with FILE_SHARE_DELETE
would cause an error right away.

Somebody should investigate this with Sysinternal's filemon, to find
out whether the MoveFile operation really succeeds. Perhaps there is
a bug in the CRT, or in Python, also.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python & Unicode decimal interpretation

[Martin v. Löwis]

Scott David Daniels wrote:
>>  >>> int(u"\N{DEVANAGARI DIGIT SEVEN}")
>> 7
> 
> OK, That much I have handled.  I am fiddling with direct-to-number
> conversions and wondering about cases like
>>>> int(u"\N{DEVANAGARI DIGIT SEVEN}" + XXX
>+ u"\N{DEVANAGARI DIGIT SEVEN}")

int() passes NULL as error mode, equalling strict. So if you get an
unencodable character, you get the UnicodeError.

> I don't really understand how the "ignore" or "something_else"
> cases get caused by python source [where they come from].  Are they
> only there for C-program access?

Neither, nor. This code is dead.

>> In the "ignore" case, no output is produced at all, for the unencodable
>> character; this is the same way that '?' would be treated (it is
>> also unencodable).
> 
> If I understand you correctly -- I can consider the digit stream to stop
> as soon as I hit a non-digit (except for handling bases 11-36).

No. In "ignore" mode, a codec doesn't stop at the unencodable character.
Instead, it skips it, continuing with the next character.

I mistakenly said that this would happen to '?' (question mark) also;
this is incorrect: PyUnicode_EncodeDecimal copies all Latin-1 characters
to the output, latin-1-encoded. So '?' would appear in the output,
even in "ignore" mode.

Handling of bases is not done in the function at all. Instead, the
callers of PyUnicode_EncodeDecimal will deal with number formats
(base, prefix, exponent syntax, etc.) They will assume ASCII
bytes.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Building Python 2.4 on machines that do not support dynamic loading

[Martin v. Löwis]

Robert McLay wrote:
> I have been trying to build python on Cray X1.  As far as I can
> tell it does not support dynamic loading.  So the question is:
> 
>How to build 2.4 without dynamic loading?

Make sure HAVE_DYNAMIC_LOADING isn't defined; configure should detect
this automatically.

> That is: can I build 2.4 where all the extensions are archived in
> libpython2.4.a as a static library?

Certainly: Edit Modules/Setup to your needs.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Detect character encoding

[Martin v. Löwis]

Martin P. Hellwig wrote:
>  From what I can remember is that they used an algorithm to create some 
> statistics of the specific page and compared that with statistic about 
> all kinds of languages and encodings and just mapped the most likely.

More hearsay: I believe language-based heuristics are common. You first
guess an encoding based on the bytes you see, then guess a language of 
the page. If you then get a lot of characters that should not appear
in texts of the language (e.g. a lot of umlaut characters in a French
page), you know your guess was wrong, and you try a different language
for that encoding. If you run out of languages, you guess a different
encoding.

Mozilla can guess the encoding if you tell it what the language is,
which sounds like a similar approach.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Detect character encoding

[Martin v. Löwis]

Diez B. Roggisch wrote:
> So cp1250 doesn't have all codepoints defined - but the others have. 
> Sure, this helps you to eliminate 1 of the three choices the OP wanted 
> to choose between - but how many texts you have that have a 129 in them?

For the iso8859 ones, you should assume that the characters in
range(128, 160) really aren't used. If you get one of these, and it is
not utf-8, it is a Windows code page.

UTF-8 can be recognized pretty reliable: even though it allows all bytes
to appear, it is very constraint in what sequences of bytes it allows.
E.g. you can't have a single byte >127 in UTF-8; you need atleast two
of them subsequent, and they need to meet more constraints.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 2.4.2 make failure in posixmodule on OSX 10.4.3

[Martin v. Löwis]

smurphus wrote:
> Any suggestions on how I can remedy this?

It would be good if you could try to understand and to
analyse these error messages; your message does not
indicate whether you have knowledge of the C language
and could analyse them or not.

For example, it is very puzzling that there is no
declaration of lchown. Can you find a declaration
for this function (there must be one, somewhere
in /usr/include)? Why is it not used?

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python 2.4.2 make failure in posixmodule on OSX 10.4.3

[Martin v. Löwis]

smurphus wrote:
> I hope that information is of some use.  Not sure where to go from
> here.

Please try
gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp
-mno-fused-madd -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I.
-I./Include  -DPy_BUILD_CORE  -c ./Modules/posixmodule.c -o
Modules/posixmodule.o --save-temps

and inspect the resulting posixmodule.i. I would expect that
it *doesn't* have the prototype for lchown.

I'm surprised though that you were suggesting (without
actually saying) that the prototype for lchown is in
sys/types.h in your system. I would have expected it in
unistd.h. Still can't check my Mac right now.

Re fstatvfs: can you verify that your system has the
fvstatvfs function? Is there a prototype declared anywhere?

Looking at

http://www.hmug.org/man/3/statvfs.php

suggests that OS X / Darwin does indeed have sys/statvfs.h.
So if you don't have it, something is broken with your
compiler setup.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python24.dll and encodings ?

[Martin v. Löwis]

Bugs wrote:
> I believe I read in a relatively recent thread that the reason 
> python24.dll is so large compared to previous releases is that all the 
> language encodings are linked into the library?

Not only that (but also). In addition, it also contains modules that
were previously implemented as separate .pyd files (_csv, _sre,
_symtable, _winreg, datetime, mmap, parser).

> Are there any plans for future releases to split the encodings out so 
> that, for example, if someone wanted to make a smaller executable using 
> py2exe without all the language encodings, they could do so?

I previously said that I would do such a thing if somebody provided a
specification other than "split out the encodings". I.e. write a PEP
that specifies how to determine whether an extension module should be
included into pythonxy.dll, and when it should not; this specification
should allow to reason about modules that haven't yet been contributed.

The PEP should be accompanied with an implementation, i.e. a set of
patches to PCbuild.

Then, the patch should be discussed, and find support in the community.

> I suppose one could always compile their own version of the python24.dll 
> but for those of us that are compiler-challanged, splitting out the 
> encodings would be nice.
> 
> Is that even feasible?

Technically, yes. However, somebody needs to take the lead.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Encoding of file names

[Martin v. Löwis]

utabintarbo wrote:
> Fredrik, you are a God! Thank You^3. I am unworthy 
> 
> I believe that may do the trick. Here is the results of running your
> code:

For all those who followed this thread, here is some more explanation:

Apparently, utabintarbo managed to get U+2592 (MEDIUM SHADE, a filled
50% grayish square) and U+2524 (BOX DRAWINGS LIGHT VERTICAL AND LEFT,
a vertical line in the middle, plus a line from that going left) into
a file name. How he managed to do that, I can only guess: most likely,
the Samba installation assumes that the file system encoding on
the Solaris box is some IBM code page (say, CP 437 or CP 850). If so,
the byte on disk would be \xb4. Where this came from, I have to guess
further: perhaps it is ACUTE ACCENT from ISO-8859-*.

Anyway, when he used listdir() to get the contents of the directory,
Windows applies the CP_ACP encoding (known as "mbcs" in Python).
For reasons unknown to me, the US and several European versions
of XP map this to \xa6, VERTICAL BAR (I can somewhat see that
as meaningful for U+2524, but not for U+2592).

So when he then applies isfile to that file name, \xa6 is mapped
to U+00A6, which then isn't found on the Samba side.

So while Unicode here is the solution, the problem is elsewhere;
most likely in a misconfiguration of the Samba server (which assumes
some encoding for the files on disk, yet the AIX application
uses a different encoding).

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Encoding of file names

[Martin v. Löwis]

Tom Anderson wrote:
> Isn't the key thing that Windows is applying a non-roundtrippable 
> character encoding?

This is a fact, but it is not a key thing. Of course Windows is
applying a non-roundtrippable character encoding. What else could it
do?

> Windows, however, maps that name to the 
> 8-bit string "double bucky blackslash vertical bar"

Only if you ask it to. There are two sets of APIs: one to apply
if you ask for byte strings (FindFirstFileA), and one to apply when you
ask for Unicode strings (FindFirstFileW).

In one case it has to convert; in the other, it doesn't.

> I don't know what Windows *should* do here. I know it shouldn't do this 
> - this leads to breaking of some very basic invariants about files and 
> directories, and so the kind of confusion utabintarbo suffered.

It always did this, and always will. Applications should stop using the
*A versions of the API. If they continue to do so, they will continue
to get bogus results in border cases.

The real issue here really is that there was a border case, when there
shouldn't be one.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Encoding of file names

[Martin v. Löwis]

Tom Anderson wrote:
>> This is a fact, but it is not a key thing. Of course Windows is 
>> applying a non-roundtrippable character encoding. What else could it do?
> 
> 
> Well, i'm no great thinker, but i'd say that errors should never pass 
> silently, and that in the face of ambiguity, one should refuse the 
> temptation to guess. So, as i said in my post, if the name couldn't be 
> translated losslessly, an error should be raised.

I believe this would not work, the way the API is structured. You do
first FindFirstFile, getting a file name and a ahandle. Then you do
FindNextFile repeatedly, passing the handle. An error of FindFirstFile
is indicated by returning an invalid handle.

So if you wanted FindFirstFile to return an error for unencodable file
names, it would not be possible to get a listing of the other files
in the directory.

FindFirstFile also gives the 8.3 file name (if present), and that is
valid without problems.

Regards,
Martin
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using ICL to compile C extensions

["Martin v. Löwis"]

[EMAIL PROTECTED] wrote:
Does anyone know how I can use "icl" (Intel C++) to compile C
extensions? I'm on Windows, and my Python is compiled using VS7.1
(binary distribution). Right now, when I run setup.py install, it uses
cl.exe (MSVC++ Toolkit 2003), and I would like to use icl because
MSVC++ 2003 does not support C99.
It should be possible to use this compiler, as long as you can link to
msvcr71.dll (i.e. as long as you have header files of the MS C library,
and an import library that links to msvcr71).
Whether you can use distutils to run the build process is a different
issue - you might need to extend distutils.
Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

Re: Wide Unicode build for Windows available somewhere?

["Martin v. Löwis"]

[EMAIL PROTECTED] wrote:
I am trying to locate a windows binary of a recent python (2.4
preferred, but not essential) with support for Unicode characters
with values greater than 0x1. I have tested the python.org
binaries and those from Activestate, both give me a traceback on
unichr(0x1) and tell me its a "narrow" build.
Wide unicode is currently not supported on Windows. A number of
internal APIs (in particular for the registry, and for the "mbcs"
codec) assume that sizeof(Py_UNICODE) is 2. Contributions are
welcome.
Even with that fixed, Pythonwin would still need a major rework
to support wide Unicode.
Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

Re: Static executable with shared modules

["Martin v. Löwis"]

Rickard Lind wrote:
Is there any way to build the python executable statically and
still be able to load modules built as shared libraries?
I'm not what "build statically" means; if you talking about
building a statically linked interpreter binary - then no,
this is not possible. At a minimum, you need to link with -ldl,
or else you cannot perform dlopen(3).
I'm trying to run python scripts on a stripped down FreeBSD (4.9)
machine which has no shared system libraries so I want to link it
statically against libc et al, but it would be nice to still be
able to load modules which were built as shared libraries.
Does that system support shared libraries? What is the API for
loading shared libraries, and finding a symbol in a dynamically-loaded
shared library, on that system?
In
particular I have a library for which I've generated a wrapper
with swig which I'd like to import.
If shared libraries are not supported, you could link the swig
module statically as well.
Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

Re: deleting from tarfile

["Martin v. Löwis"]

Mark McEahern wrote:
It doesn't appear so.  A workaround, of course, is to create a new file 
with the subset of files from the old file:
That is actually the *only* way to do that. tarfiles cannot be "sparse",
in the sense that parts of the file can be marked as deleted. So in
order to delete a file, you have to copy the entire tarfile, and skip
the file you want to delete - whether you do this yourself, or whether
tarfile.py does it for you.
Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

Re: mutable string?

["Martin v. Löwis"]

Torsten Mohr wrote:
is there some string class that i can change in place,
like perls strings?
array.array is mutable. You can use the 'c' code if
you want an array of characters.
Is it possible to do some regex replacement functions
that would even change its length?
You can't use re.sub array, but you can use re.search,
and you can perform slice assigment.
Can i append to this string?
Yes.
Is there some wrapper for C++ STL string class (if that
would make sense)?
It would be possible, but I cannot see why one would want
to do that.
Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

Re: why are some types immutable?

["Martin v. Löwis"]

Torsten Mohr wrote:
reading the documentation (and also from a hint from this NG)
i know now that there are some types that are not mutable.
But why is it this way?
There are various reasons, some apply for some types, and
some for others:
- immutable objects are hashable - their hash value will not
  change during their life time (if it is a container, the
  contained objects also need to be hashable). Mutable
  types are typically not hashable.
  A type needs to be hashable to act as a dictionary key.
- immutable objects can be shared, allowing the same
  reference to be used in multiple places. For mutable
  objects, one often needs to make a copy of the object
  before storing it.
- immutable types can often be implemented more efficiently.
  For examples, the length of a tuple is fixed, as tuples
  are immutable. Therefore, memory management for tuples
  is simpler (they only require one memory block, instead
  of two, as lists do).
- for some types, the expectation of immutability is so
  common that people would complain massively if they were
  mutable. This, in particular, applies to numbers - people
  expect that after
  x = 7
  the variable x keeps the value, and that the "7" object
  cannot suddenly change its value to 8.
From an overhead point of view i think it is not optimal,
for example for a large string it could be much faster to
have it changed in place, not generating a new one for
every step in a change.
For strings, the following points apply; as a net result,
the overhead is *less* than it would be if strings were
mutable:
- strings must be hashable, so they can be used as
  a dictionary key. If you would modify, say,
  len.func_name, then newly imported code could not
  find the len function anymore.
  It would be possible to solve this by introducing
  an additional immutable identifier type, but then
  you would need to create copies of strings (to
  create identifier object). Many applications might
  break which currently use strings as dictionary
  keys.
- strings are currently assumed to be shareable.
  You pass strings to functions which then store
  the strings in global variables, object attributes,
  etc. In all these places, copies would need to be
  made if strings were mutable, since the caller
  might chose to modify the string afterwards.
- immutable strings consume less memory than
  mutable strings would, because a string is made
  up of just one memory block.
Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

Re: Static executable with shared modules

["Martin v. Löwis"]

Rickard Lind wrote:
I'll be more specific: when I build python 2.3.4 on FreeBSD 4.9,
the interpreter binary is linked against three shared libraries: 
libutil.so.3, libm.so.2 and libc_r.so.4. Now, these libraries are
not present on the TARGET system (which is distinct from the build
system, but based on the same version of FreeBSD) so I add "-static"
to LDFLAGS.  This produces an interpreter that runs on the target
system (no dependency on shared libc etc) but it also cannot load
modules compiled as shared libraries. Man page for dlopen(3) says
it is located in libc (and consquently there seems to be no libdl),
and anyway I'd expect to get a link error if the dl* functions were
not present. What I DO get is an ImportError exception.
Most likely, the static libc.a does not provide dlopen (contrary
to what the man page says). Python's configure detects that you
don't have dlopen (when linking with -static), and concludes that
dynamic loading of modules is not supported on your system.
Then, during import, it checks built-in modules, module.py, module.pyc,
and finds neither - it then reports an import error.
To confirm this theory, have a look at pyconfig.h, and check
whether HAVE_DLOPEN and/or HAVE_DYNAMIC_LOADING are defined;
I expect that neither is defined.
At present I see no other option than to link the modules into the
interpreter which is very inconvenient since I'll have to rebuild
the every time a module changes :-(
The other option, clearly, is to move the shared libraries to the
target system along with the interpreter binary (assuming the target
system supports shared libraries).
If the target system does not have a dlopen, and the static libc.a
does not have dlopen, either, there is nothing you can do except
to use a different operating system.
Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

Re: xml parsing escape characters

["Martin v. Löwis"]

Luis P. Mendes wrote:
I get the following result:

http://www..";>
~  
Most likely, this result is correct, and your document
really does contain
  

I don't get any elements.  But, if I access the same url via a browser,
the result in the browser window is something like:
http://www..";>
~  
Most likely, your browser is incorrect (or atleast confusing), and
renders < as "<", even though this is not markup.
I already browsed the web, I know it's about the escape characters, but
I didn't find a simple solution for this.
Not sure what "this" is. AFAICT, everything works correctly.
Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

Re: rotor replacement

["Martin v. Löwis"]

Paul Rubin wrote:
Some countries have laws about cryptography software (against some
combination of export, import, or use).  The Python maintainers didn't
want to deal with imagined legal hassles that might develop from
including good crypto functions in the distribution.  Then it became
obvious that the same imagined hassles could also befall the rotor
module, so that was removed.
Do you know this for a fact? The PSF does comply with the U.S. American
export procedures for crypto code, and reports the crypto code in
Python appropriately to BXA.
Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

Re: xml parsing escape characters

["Martin v. Löwis"]

Luis P. Mendes wrote:
with:DataSetNode = stringNode.childNodes[0]
print DataSetNode.toxml()
I get:

~  
~439
~  

___-
so far so good, but when I issue the command:
print DataSetNode.childNodes[0]
I get:
IndexError: tuple index out of range
Why the error, and why does it return a tuple?
The DataSetNode has no children, because it is not
an Element node, but a Text node. In XML, an element
is denoted by
  ...
and *not* by
  ...
The latter is just a single string, represented
in XML as a Text node. It does not give you any
hierarchy whatsoever.
As a text node does not have any children, its
childNode members is a empty tuple; accessing
that tuple gives you an IndexError.
Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

Re: xml parsing escape characters

["Martin v. Löwis"]

Irmen de Jong wrote:
The unescaping is usually done for you by the xml parser that you use.
Usually, but not in this case. If you have a text that looks like
XML, and you want to put it into an XML element, the XML file uses
< and >. The XML parser unescapes that as < and >. However, it
does not then consider the < and > as markup, and it shouldn't.
Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

Re: xml parsing escape characters

["Martin v. Löwis"]

Irmen de Jong wrote:
Usually, but not in this case. If you have a text that looks like
XML, and you want to put it into an XML element, the XML file uses
< and >. The XML parser unescapes that as < and >. However, it
does not then consider the < and > as markup, and it shouldn't.

That's also what I said?
You said it in response to
>>> All the behaviour you are seeing is a consequence of this. You need
>>> to unescape the contents of the  tag to be able to treat it 
>>> as structured XML.

In that context, I interpreted
>> The unescaping is usually done for you by the xml parser that you
>> use.
as "The parser should have done what you want; if the parser didn't,
that is is bug in the parser".
The OP probably wants to feed that to a new xml parser instance
to process it as markup.
Or perhaps the way the original XML document is constructed is
flawed.
Either of these, indeed - probably the latter.
Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

Re: xml parsing escape characters

["Martin v. Löwis"]

Luis P. Mendes wrote:
When I access the url via the Firefox browser and look into the source
code, I also get:


~  
~439
~  

Please do try to understand what you are seeing. This is crucial for
understanding what happens.
You may have the understanding that XML can be represented as a tree.
This would be good - if not, please read a book that explains why
XML can be considered as a tree.
In the tree, you have inner nodes, and leaf nodes. For example,
the document

  Hello
  World

has 5 nodes (ignoring whitespace content):
Element:a  Element:b  Text:"Hello"
   |
   \-- Element:c  Text:"World"
So the leaf nodes are typically Text nodes (unless you
have an empty element). Your document has this structure:
Element:string  Text:"""
   
  439
  
"""
So the ***TEXT*** contains the letter "<", just like it contains
the letters "O" and "r". There IS no element Order in your document,
no matter how hard you look.
If you want a DataSet *element* in your document, it should
read

 
  
   439
  

As this is the document you apparently want to process, complain
to whoever gave you that other document.
should I take the contents of the string tag that is text and replace
all '<' with '<' and '>' with '>' and then read it with xml.minidom?
No. We still don't know what you want to achieve, so it is difficult to
advise you what to do. My best advise is that whoever generates the XML
document should fix it.
or should I use another parser that accomplishes the task with no need
to replace the escaped characters?
No. The parser is working correctly.
The document you got can also be interpreted as containing another
XML document as a text. This is evil, but apparently people are doing
it, anyway. If you really want that embedded document, you need
first to extract it.
To see what I mean, do
print DataSetNode.data
The .data attribute gives you the string contents of
a text node. You could use this as an XML document, and
parse it again to an XML parser. This would be ugly,
but might be your only choice if the producer of the
document is unwilling to adjust.
Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list

Re: ElementTree cannot parse UTF-8 Unicode?

["Martin v. Löwis"]

Jarek Zgoda wrote:
So why are there non-UNICODE versions of wxPython??? To save memory or
something???

Win95, Win98, WinME have problems with unicode. 
This problem can be solved - on W9x, wxPython would have to
pass all Unicode strings to WideCharToMultiByte, using
CP_ACP, and then pass the result to the API function.
Regards,
Martin
--
http://mail.python.org/mailman/listinfo/python-list