Re: [Python-Dev] [C++-sig] GCC version compatibility
On Tue, Jul 12, 2005 at 01:07:56AM +0200, "Martin v. Löwis" wrote: > Christoph Ludwig wrote: > > Yes, but on ELF/Linux the default configuration should be --without-cxx > > in the first place. If the build instructions make it sufficiently clear > > that > > you should prefer this configuration whenever possible then this should be a > > non-issue on platforms like ELF/Linux. > > Some users will complain about this. Specifying --without-cxx also > causes configure not to look for a C++ compiler, meaning that distutils > won't know what the C++ compiler is, meaning that it will link extension > modules with the C compiler instead. If I understood Dave Abraham's reply somewhere above in this thread correctly then you can build different C++ extension modules with different C++ compilers on ELF/Linux. (I don't have the time right now to actually try it, sorry.) There is no need to fix the C++ compiler as soon as python is built. If distutils builds C++ extensions with the C compiler then I consider this a bug in distutils because it is unlikely to work. (Unless the compiler can figure out from the source file suffixes in the compilation step *and* some info in the object files in the linking step that it is supposed to act like a C++ compiler. None of the compilers I am familiar with does the latter.) distutils should rather look for a C++ compiler in the PATH or explicitly ask the user to specify the command that calls the C++ compiler. It is different if --with-cxx= was used. I agree that in this case distutils should use to build C++ extensions. (distutils does not behave correctly when building C++ extensions anyway. It calls the C compiler to compile the C++ source files and passes options that gcc accepts only in C mode. The compiler version I am using is docile and only issues warnings. But these warnings are unnecessary and and I would not blame gcc if the next compiler release refused to compile C++ sources if the command line contains C specific options. But the distutils mailing list is a better place to bring this eventually up, I guess.) Regards Christoph -- http://www.informatik.tu-darmstadt.de/TI/Mitarbeiter/cludwig.html LiDIA: http://www.informatik.tu-darmstadt.de/TI/LiDIA/Welcome.html ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
Hi Neil,
>>>2) Return unicode when the text can not be represented in ASCII. This
>>>will cause a change of behaviour for existing code which deals with
>>>non-ASCII data.
>>
>>+1 on this one (s/ASCII/Python's default encoding).
>
>
>I assume you mean the result of sys.getdefaultencoding() here.
Yes.
The default encoding is the encoding that Python assumes when
auto-converting a string to Unicode. It is normally set to ASCII,
but a user may want to use a different encoding.
However, we've always made it very clear that the user is on his
own when chainging the ASCII default to something else.
> Unless much of the Python library is modified to use the default
> encoding, this will break. The problem is that different implicit
> encodings are being used for reading data and for accessing files.
> When calling a function, such as open, with a byte string, Python
> passes that byte string through to Windows which interprets it as
> being encoded in CP_ACP. When this differs from
> sys.getdefaultencoding() there will be a mismatch.
As I said: code pages are evil :-)
>Say I have been working on a machine set up for Australian English
> (or other Western European locale) but am working with Russian data so
> have set Python's default encoding to cp1251. With this simple script,
> g.py:
>
> import sys
> print file(sys.argv[1]).read()
>
>I process a file called '€.txt' with contents "European Euro" to produce
>
> C:\zed>python_d g.py €.txt
> European Euro
>
>With the proposed modification, sys.argv[1] u'\u20ac.txt' is
> converted through cp1251
Actually, it is not: if you pass in a Unicode argument to
one of the file I/O functions and the OS supports Unicode
directly or at least provides the notion of a file system
encoding, then the file I/O should use the Unicode APIs
of the OS or convert the Unicode argument to the file system
encoding. AFAIK, this is how posixmodule.c already works
(more or less).
I was suggesting that OS filename output APIs such as os.listdir()
should return strings, if the filename matches the default
encoding, and Unicode, if not.
On input, file I/O APIs should accept both strings using
the default encoding and Unicode. How these inputs are then
converted to suit the OS is up to the OS abstraction layer, e.g.
posixmodule.c.
Note that the posixmodule currently does not recode string
arguments: it simply passes them to the OS as-is, assuming
that they are already encoded using the file system encoding.
Changing this is easy, though: instead of using the "et"
getargs format specifier, you'd have to use "es". The latter
recodes strings based on the default encoding assumption to
whatever other encoding you specify.
> to '\x88.txt' as the Euro is located at 0x88
> in CP1251. The operating system is then asked to open '\x88.txt' which
> it interprets through CP_ACP to be u'\u02c6.txt' ('ˆ.txt') which then
> fails. If you are very unlucky there will be a file called 'ˆ.txt' so
> the call will succeed and produce bad data.
>
>Simulating with str(sys.argvu[1]):
>
> C:\zed>python_d g.py €.txt
> Traceback (most recent call last):
> File "g.py", line 2, in ?
> print file(str(sys.argvu[1])).read()
> IOError: [Errno 2] No such file or directory: '\x88.txt'
See above: this is what I'd consider a bug in posixmodule.c
>>-1: code pages are evil and the reason why Unicode was invented
>>in the first place. This would be a step back in history.
>
>
>Features used to specify files (sys.argv, os.environ, ...) should
> match functions used to open and perform other operations with files
> as they do currently. This means their encodings should match.
Right. However, most of these APIs currently either don't
make any assumption on the strings contents and simply pass
them around, or they assume that these strings use the file
system encoding - which, like in the example you gave above,
can be different from the default encoding.
To untie this Gordian Knot, we should use strings and Unicode
like they are supposed to be used (in the context of text
data):
* strings are fine for text data that is encoded using
the default encoding
* Unicode should be used for all text data that is not
or cannot be encoded in the default encoding
Later on in Py3k, all text data should be stored in Unicode
and all binary data in some new binary type.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, Jul 12 2005)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free !
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
Re: [Python-Dev] [C++-sig] GCC version compatibility
Christoph Ludwig wrote: > If I understood Dave Abraham's reply somewhere above in this thread correctly > then you can build different C++ extension modules with different C++ > compilers on ELF/Linux. (I don't have the time right now to actually try it, > sorry.) There is no need to fix the C++ compiler as soon as python is built. There is, somewhat: how do you know the name of the C++ compiler? > If distutils builds C++ extensions with the C compiler then I consider this a > bug in distutils because it is unlikely to work. (Unless the compiler can > figure out from the source file suffixes in the compilation step *and* some > info in the object files in the linking step that it is supposed to act like a > C++ compiler. None of the compilers I am familiar with does the latter.) > distutils should rather look for a C++ compiler in the PATH or explicitly ask > the user to specify the command that calls the C++ compiler. How should it do that? The logic is quite involved, and currently, distutils relies on configure to figure it out. If you think this should be changed, please contribute a patch. > (distutils does not behave correctly when building C++ extensions anyway. It > calls the C compiler to compile the C++ source files and passes options that > gcc accepts only in C mode. The compiler version I am using is docile and only > issues warnings. But these warnings are unnecessary and and I would not blame > gcc if the next compiler release refused to compile C++ sources if the command > line contains C specific options. But the distutils mailing list is a better > place to bring this eventually up, I guess.) The best way to "bring this up" is to contribute a patch. "Bringing it up" in the sense of sending an email message to some mailing list likely has no effect whatsoever. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
Tim Peters <[EMAIL PROTECTED]> writes: > [Michael Hudson] >> --with-fpectl, for example. Does anyone lurking here actually use >> that, know what it does and require the functionality? Inquiring >> minds want to know. > > I know what it intends to do: Surprise! > fpectlmodule.c intends to enable the HW FPU divide-by-0, overflow, > and invalid operation traps; if any of those traps trigger, raise > the C-level SIGFPE signal; and convert SIGFPE to a Python-level > FloatingPointError exception. The comments in pyfpe.h explain this > best. But do you use it? I know what it intends to do too, but I don't use it. The questions I asked were in the order they were for a reason. Cheers, mwh -- If you are anal, and you love to be right all the time, C++ gives you a multitude of mostly untimportant details to fret about so you can feel good about yourself for getting them "right", while missing the big picture entirely -- from Twisted.Quotes ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible context managers in stdlib
Skip Montanaro wrote: > After seeing so many messages about "with" statements my eyes began to glaze > over, so I stopped following that thread. Then I saw mention of "context > manager" with no reference to any PEPs or to the with statement to provide > context. The main outcome of the PEP 343 terminology discussion was some proposed documentation I put on the Sourceforge patch tracker ([1]). The patch is currently assigned to Raymond (since he started the terminology discussion) but any other reviews would be welcome. Since SF currently doesn't want to play, and the proposed documentation isn't that long, I've included the latest version below for anyone who wants to read it. > None of the context-providing messages seemed to have been indexed > by Google when I checked, so searching for "Python context manager" failed > to return anything useful. Hence the post. Google appears to have spidered the list archives some time today, so anyone else doing the same search should get some relevant hits. Cheers, Nick. [1] http://www.python.org/sf/1234057 == With Statements and Context Management A frequent need in programming is to ensure a particular action is taken after a specific section of code has been executed (such as closing a file or releasing a lock). Traditionally, this is handled using 'try'/'finally' statements. However, that approach can lead to the reproduction of non-trivial amounts of boilerplate whenever the action needs to be invoked. A simpler way to achieve this in Python is to use the 'with' statement along with the appropriate context manager. Context managers define an action which is taken to enter the context and a second action to exit the context (usually restoring the environment that existed before the context was entered). The 'with' statement ensures that the context is entered and exited at the appropriate times (that is, before and after the execution of the suite contained in the 'with' statement). The precise behaviour of the 'with' statement is governed by the supplied context manager - an object which supports the context management protocol. This protocol consists of two methods: __enter__(self): Context managers use this method to enter the desired context before the execution of the contained suite. This method is called without arguments before execution of the contained suite starts. If the 'as' clause of the 'with' statement is used, the value returned from this method is assigned to the specified target. Many context managers will return self from this method, but returning a different object may make sense for some managers (for example, see the 'closing' suite manager described below). __exit__(self, exc_type, exc_value, exc_traceback): Context managers use this method to exit the context after execution of the contained suite. This method is called after execution of the contained suite is completed. If execution completed due to an exception, the details of that exception are passed as arguments. Otherwise, all three arguments are set to None. If exception details are passed in, and this method returns without incident, then the original exception continues to propagate. Otherwise, the exception raised by this method will replace the original exception. Using Contexts to Manage Resources The simplest use of context management is to strictly control the handling of key resources (such as files, generators, database connections, synchronisation locks). These resource managers will generally acquire the resource in their __enter__ method, although some resource managers may accept the resource to be managed as an argument to the constructor or acquire it during construction. Resource managers will then release the resource in their __exit__ method. For example, the following context manager allows prompt closure of any resource with a 'close' method (e.g. a generator or file): class closing(object): def __init__(self, resource): self.resource = resource def __enter__(self): return self.resource def __exit__(self, *exc_info): self.resource.close() with closing(my_generator()) as g: # my_generator() is assigned to g via call to __enter__() for item in g: print item # g is closed as the with statement ends Some resources (such as threading.Lock) support the context management protocol natively, allowing them to be used directly in 'with' statements. The meaning of the established context will depend on the specific resource. In the case of threading.Lock, the lock is acquired by the __enter__ method, and released by the __exit__ method. with the_lock: # Suite is executed with the_lock held # the_lock is released as the with statement e
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFEfor review)
M A Lemburg writes: > we should use strings and Unicode > like they are supposed to be used (in the context of text > data): > > * strings are fine for text data that is encoded using > the default encoding > > * Unicode should be used for all text data that is not > or cannot be encoded in the default encoding > > Later on in Py3k, all text data should be stored in Unicode > and all binary data in some new binary type. Wow. That is the most succinct and clear explanation of how to use unicode in Python that I think I've ever heard. It might even be simple enough for _me_ to understand it! I think I'm going to go frame this and have it posted in my cubical. -- Michael Chermside ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFEfor review)
QOTF candidate; should add that the default encoding is usually ASCII. On 7/12/05, Michael Chermside <[EMAIL PROTECTED]> wrote: > M A Lemburg writes: > > we should use strings and Unicode > > like they are supposed to be used (in the context of text > > data): > > > > * strings are fine for text data that is encoded using > > the default encoding > > > > * Unicode should be used for all text data that is not > > or cannot be encoded in the default encoding > > > > Later on in Py3k, all text data should be stored in Unicode > > and all binary data in some new binary type. > > Wow. That is the most succinct and clear explanation of how to > use unicode in Python that I think I've ever heard. It might > even be simple enough for _me_ to understand it! I think I'm > going to go frame this and have it posted in my cubical. > > -- Michael Chermside > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
Nobody uses it. It should be ripped out. If someone disagrees, let them speak up. On 7/12/05, Michael Hudson <[EMAIL PROTECTED]> wrote: > Tim Peters <[EMAIL PROTECTED]> writes: > > > [Michael Hudson] > >> --with-fpectl, for example. Does anyone lurking here actually use > >> that, know what it does and require the functionality? Inquiring > >> minds want to know. > > > > I know what it intends to do: > > Surprise! > > > fpectlmodule.c intends to enable the HW FPU divide-by-0, overflow, > > and invalid operation traps; if any of those traps trigger, raise > > the C-level SIGFPE signal; and convert SIGFPE to a Python-level > > FloatingPointError exception. The comments in pyfpe.h explain this > > best. > > But do you use it? I know what it intends to do too, but I don't use > it. The questions I asked were in the order they were for a reason. > > Cheers, > mwh > > -- > If you are anal, and you love to be right all the time, C++ >gives you a multitude of mostly untimportant details to fret about >so you can feel good about yourself for getting them "right", >while missing the big picture entirely -- from Twisted.Quotes > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [C++-sig] GCC version compatibility
On 7/12/05, Christoph Ludwig <[EMAIL PROTECTED]> wrote: > If distutils builds C++ extensions with the C compiler then I consider this a > bug in distutils because it is unlikely to work. (Unless the compiler can > figure out from the source file suffixes in the compilation step *and* some > info in the object files in the linking step that it is supposed to act like a > C++ compiler. None of the compilers I am familiar with does the latter.) > distutils should rather look for a C++ compiler in the PATH or explicitly ask > the user to specify the command that calls the C++ compiler. You practically always have to use --compiler with distutils when building C++ extensions anyhow, and even then it rarely does what I would consider 'The Right Thing(tm)'. The problem is the distutils core assumption that you want to build extension modules with the same compiler options that you built Python with, is in many cases the wrong thing to do for C++ extension modules, even if you built Python with --with-cxx. This is even worse on windows where the MSVC compiler, until very recently, was crap for C++, and you really needed to use another compiler for C++, but Python was always built using MSVC (unless you jumped through hoops of fire). The problem is that this is much more complicated than it seems - you can't just ask the user for the C++ compiler, you really need to provide an abstraction layer for all of the compiler and linker flags, so that a user could specify what those flags are for their compiler of choice. Of course, once you've done that, the user might as well have just written a new Compiler class for distutils, which wouldn't pay any attention to how Python was built (other than where Python.h is). -- Nick ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Linux Python linking with G++?
[Michael Hudson] >>> --with-fpectl, for example. Does anyone lurking here actually use >>> that, know what it does and require the functionality? Inquiring >>> minds want to know. [Tim, explains what it intends to do] >> ... [Michael] > But do you use it? I know what it intends to do too, Then why ask what it does? You're asking a lot of questions lately that you're irked by when you get an answer <0.9 wink>. > but I don't use it. The questions I asked were in the order they were > for a reason. No, I don't use it -- last I looked seriously (which was years ago), it didn't work on Windows. I don't know of anyone who does use it. The functionality is highly desired by some numeric programmers, in some parts of some of their apps, although they'd like finer control (like don't gripe about divide-by-0 but do gripe about overflow in function A, and don't gripe about anything in function B, and only gripe about invalid operation in the 23rd line of function C). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible context managers in stdlib
FWIW, I've updated PEP 343 to use @contextmanager and class ContextWrapper. Please proofread. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible context managers in stdlib
"Nick Coghlan" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > The main outcome of the PEP 343 terminology discussion was some proposed > documentation I put on the Sourceforge patch tracker ([1]). Is this a proposal for the Language Reference manual? > [1] http://www.python.org/sf/1234057 > == > With Statements and Context Management > > A frequent need in programming ... > > A simpler way to achieve this in Python is to use the 'with' statement > along with the appropriate context manager. Somewhere about here we need the syntax itself. > Context managers define an... > the contained suite starts. If the 'as' clause of the 'with' Else this does not mean much. ... > The simplest use of context management is to strictly control the > handling of key resources (such as files, generators, database > connections, synchronisation locks). I have a little trouble seeing generators (as opposed to iterables) as resources needing management. > For example, the following context manager allows prompt closure of > any resource with a 'close' method (e.g. a generator or file): And I was not aware that they had close methods. You mean a iterable (not just a file) with both an associated generator and a close? Or are generators gaining close methods (which make no sense to me). Or are you using 'generator' in a different sense? > class closing(object): > def __init__(self, resource): > self.resource = resource > > def __enter__(self): > return self.resource > > def __exit__(self, *exc_info): > self.resource.close() > > > with closing(my_generator()) as g: > # my_generator() is assigned to g via call to __enter__() > for item in g: > print item > # g is closed as the with statement ends To me, this should be with closing(my_iterable())... with 'for' calling g.__iter__ to get the iterator that is possibly a generator. Otherwise, I don't understand it. The rest is pretty clear. Terry J. Reedy ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Terminology for PEP 343
Probably late in the game, esp. for an outsider, but I read the terminology discussion with interest. FWIW, I do like Philip's use of context, though I feel that it is a very generic word that may clash with many application-level classes... For that reason, I also liked "scope" a lot, though it was an... expension of that term's usual meaning beyond namespaces. Anyway, what really struck me all along is that, when reading the keyword "with", I always felt that I would replace it with "within", which imho fits the context/scope terminology better. Thus "within" a "context", we do certain actions... which are fenced with __begincontext and __endcontext. (Oh, yes, fences... What was the original precise computer science meaning of that word, again?) Cheers, Marc-Antoine ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Possible context managers in stdlib
Terry Reedy wrote: > "Nick Coghlan" <[EMAIL PROTECTED]> wrote in message > news:[EMAIL PROTECTED] > >>The main outcome of the PEP 343 terminology discussion was some proposed >>documentation I put on the Sourceforge patch tracker ([1]). > > Is this a proposal for the Language Reference manual? No - it's for an entry in the Library reference under 'built-in types', as a sibling to the current documentation of the iteration protocol. The 'with' statement itself would have to be documented along with the rest of the grammar. >>A simpler way to achieve this in Python is to use the 'with' statement >>along with the appropriate context manager. > > Somewhere about here we need the syntax itself. I'm not sure. We don't reproduce the 'for' loop syntax in the documentation of iterators, so should we reproduce the 'with' statement syntax in the documentation of context managers? Again, modelling on the existing documentation of the iteration protocol, I would expect the statement syntax to be introduced in the tutorial (e.g. as part of Section 8.6, where try/finally is introduced). >>Context managers define an... >> the contained suite starts. If the 'as' clause of the 'with' > > > Else this does not mean much. > ... > >>The simplest use of context management is to strictly control the >>handling of key resources (such as files, generators, database >>connections, synchronisation locks). > > I have a little trouble seeing generators (as opposed to iterables) as > resources needing management. PEP 342 adds this, in order to allow 'yield' inside tyr/finally blocks. > > >>For example, the following context manager allows prompt closure of >>any resource with a 'close' method (e.g. a generator or file): > > > And I was not aware that they had close methods. You mean a iterable (not > just a file) with both an associated generator and a close? Or are > generators gaining close methods (which make no sense to me). Or are you > using 'generator' in a different sense? Sorry - these docs assume PEP 342 has been implemented, so generator's have close() methods. I was trying to steer clear of files, since we don't know yet whether there is going to be an "opening" or "closing" implementation in the standard library, or whether files will become context managers. The latter is my preference, but Guido didn't seem too keen on the idea last time it was brought up. > > >> class closing(object): >> def __init__(self, resource): >> self.resource = resource >> >> def __enter__(self): >> return self.resource >> >> def __exit__(self, *exc_info): >> self.resource.close() >> >> >>with closing(my_generator()) as g: >># my_generator() is assigned to g via call to __enter__() >>for item in g: >>print item >># g is closed as the with statement ends > > > To me, this should be with closing(my_iterable())... with 'for' calling > g.__iter__ to get the iterator that is possibly a generator. Otherwise, I > don't understand it. > > The rest is pretty clear. > > Terry J. Reedy > > > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ncoghlan%40email.com > -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding the 'path' module (was Re: Some RFE for review)
Hi Marc-Andre,
> >With the proposed modification, sys.argv[1] u'\u20ac.txt' is
> > converted through cp1251
>
> Actually, it is not: if you pass in a Unicode argument to
> one of the file I/O functions and the OS supports Unicode
> directly or at least provides the notion of a file system
> encoding, then the file I/O should use the Unicode APIs
> of the OS or convert the Unicode argument to the file system
> encoding. AFAIK, this is how posixmodule.c already works
> (more or less).
Yes it is. The initial stage is reading the command line arguments.
The proposed modification is to change behaviour when constructing
sys.argv, os.environ or when calling os.listdir to "Return unicode
when the text can not be represented in Python's default encoding". I
take this to mean that when the value can be represented in Python's
default encoding then it is returned as a byte string in the default
encoding.
Therefore, for the example, the code that sets up sys.argv has to
encode the unicode command line argument into cp1251.
> On input, file I/O APIs should accept both strings using
> the default encoding and Unicode. How these inputs are then
> converted to suit the OS is up to the OS abstraction layer, e.g.
> posixmodule.c.
This looks to me to be insufficiently compatible with current
behaviour whih accepts byte strings outside the default encoding.
Existing code may call open("€.txt"). This is perfectly legitimate
current Python (with a coding declaration) as "€.txt" is a byte string
and file systems will accept byte string names. Since the standard
default encoding is ASCII, should such code raise UnicodeDecodeError?
> Changing this is easy, though: instead of using the "et"
> getargs format specifier, you'd have to use "es". The latter
> recodes strings based on the default encoding assumption to
> whatever other encoding you specify.
Don't you want to convert these into unicode rather than another
byte string encoding? It looks to me as though the "es" format always
produces byte strings and the only byte string format that can be
passed to the operating system is the file system encoding which may
not contain all the characters in the default encoding.
Neil
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [C++-sig] GCC version compatibility
Nicholas Bastin wrote: > You practically always have to use --compiler with distutils when > building C++ extensions anyhow, and even then it rarely does what I > would consider 'The Right Thing(tm)'. I see. In that case, I think something should be done about distutils as well (assuming somebody volunteers): it would be best if this worked in the usual cases, with some easy way for the setup.py author to indicate the preferences. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
