date:20090928

Re: [Python-Dev] Fuzziness in io module specs - PEP update proposition V2

2009-09-28 Thread Pascal Chambon


Antoine Pitrou a écrit :

Hello,

  

So here is the proposed semantic, which matches established conventions:

*IOBase.truncate(n: int = None) -> int*


[...]

I still don't think there is a sufficient benefit in breaking 
compatibility. If you want the file pointer to remain the same, you can 
save it first and restore it afterwards manually.


  
Sure, but won't this truncate become some kind of a burden for py3k, if 
it's twice misleading (it's not a real truncation since it can extend 
the file, and it's not even a truncation or resizing in posix/win32 
style, since the filepointer is moved) ?
Since it was an undocumented behaviour, and py3k doesn't seem to be 
present yet in production environments (or is it ?), I'd promote this 
late-but-maybe-not-too-late change.
But if the consensus prefers the current behaviour, well, it'll be OK to 
me too, as long as it's sufficiently documented and advertised.



*Propositions of doc update*



Please open tracker issues for these kinds of suggestions.
  
Is the tracker Ok for simple suggestions too ? I thought it was rather 
for obvious bugfixes, and to-be-discused propositions had better be in 
mailing-lists... OK then, I'll open bugtracker issues for these. B-)





Instead of "than size", perhaps "than n".


Whoups, indeed >_<
Actually the signature would rather be:
*IOBase.truncate(size: int = None) -> int*
And I forgot to mention that truncate returns the new file size 
(according to the current PEP)...




Should an exception be raised if start and/or end are out of range?
I'd advocate it yep, for the sake of "explicit errors". However, should 
it be a ValueError (the ones io functions normally use) or an IndexError 
(which is technically more accurate, but might confuse the user) ?



Regards,
Pascal




___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] please consider changing --enable-unicode default to ucs4

2009-09-28 Thread M.-A. Lemburg

Zooko O'Whielacronx wrote:
> Folks:
> 
> I'm sorry, I think I didn't make my concern clear.  My users, and lots
> of other users, are having a problem with incompatibility between
> Python binary extension modules.  One way to improve the situation
> would be if the Python devs would use their "bully pulpit" -- their
> unique position as a source respected by all Linux distributions --
> and say "We recommend that Linux distributions use UCS4 for
> compatibility with one another".  This would not abrogate anyone's
> ability to choose their preferred setting nor, as far as I can tell,
> would it interfere with the ongoing development of Python.

-1

Please note that we did not choose to ship Python as UCS4 binary
on Linux - the Linux distributions did.

The Python default is UCS2 for a good reason: it's a good trade-off
between memory consumption, functionality and performance.

As already mentioned, I also don't understand how the changing
the Python default on Linux would help your users in any way -
if you let distutils compile your extensions, it's automatically
going to use the right Unicode setting for you (as well as your
users).

Unfortunately, this automatic support doesn't help you when
shipping e.g. setuptools eggs, but this is a tool problem,
not one of Python: setuptools completely ignores the fact
that there are two ways to build Python.

I'd suggest you ask the tool maintainers to adjust their tools
to support the Python Unicode option.

> Here are the details:
> 
> I'm the maintainer of several Python packages.  I work hard to make it
> easy for users, even users who don't know anything about Python, to
> use my software.  There have been many pain points in this process and
> I've spent a lot of time on it for about three years now working on
> packaging, including the tools such as setuptools and distutils and
> the new "distribute" tool.  Python packaging has been improving during
> these years -- things are looking up.
> 
> One of the remaining pain points is that I can distribute binaries of
> my Python extension modules for Windows or Mac, but if I distribute a
> binary Python extension module on Linux, then if the user has a
> different UCS2/UCS4 setting then they won't be able to use the
> extension module.  The current de facto standard for Linux is UCS4 --
> it is used by Debian, Ubuntu, Fedora, RHEL, OpenSuSE, etc. etc..  The
> vast majority of Linux users in practice have UCS4, and most binary
> Python modules are compiled for UCS4.
> 
> That means that a few folks will get left out.  Those folks, from my
> experience, are people who built their python executable themselves
> without specifying an override for the default, and the smaller Linux
> distributions who insist on doing whatever upstream Python devs
> recommend instead of doing whatever the other Linux distros are doing.
>  One of the data points that I reported was a Python interpreter that
> was built locally on an Ubuntu server.  Since the person building it
> didn't know to override the default setting of --enable-unicode, he
> ended up with a Python interpreter built for UCS2, even though all the
> Python extension modules shipped by Ubuntu were built with UCS4.

People building their own Python version will usually also build
their own extensions, so I don't really believe that the above
scenario is very common.

Also note that Python will complain loudly when you try to load
a UCS2 extension in a UCS4 build and vice-versa. We've made sure
that any extension using the Python Unicode C API has to be built
for the same UCS version of Python. This is done by using different
names for the C APIs at the C level.

> These are not isolated incidents.  The following google searches
> suggest that a number of people spend time trying to figure out why
> Python extension modules fail on their linux systems:
> 
> http://www.google.com/search?q=PyUnicodeUCS4_FromUnicode+undefined+symbol
> http://www.google.com/search?q=+PyUnicodeUCS2_FromUnicode+undefined+symbol
> http://www.google.com/search?q=_PyUnicodeUCS2_AsDefaultEncodedString+undefined+symbol

Perhaps we should add a FAQ entry for these linker errors
(which are caused by the mentioned C API changes to prevent
mixing UCS version) ?!

Here's a quick way to determine you Python Unicode build type:

python -c "import sys;print((sys.maxunicode<66000)and'UCS2'or'UCS4')"

Perhaps we should include this info as well as an 32/64-bit indicator
and the processor type in the Python startup line:

# python
Python 2.6 (r26:66714, Feb  3 2009, 20:49:49, UCS4, 64-bit, x86_64)
[GCC 4.3.2 [gcc-4_3-branch revision 141291]] on linux2
Type "help", "copyright", "credits" or "license" for more information.

This would help users find the right binaries to install as
extension.

> Another data point is the Mandriva Linux distribution.  It is probably
> much smaller than Debian, Ubuntu, or RedHat, but it is still one of
> the major, well-known distributions.  I requested of the Python
> maintai

Re: [Python-Dev] please consider changing --enable-unicode default to ucs4

2009-09-28 Thread M.-A. Lemburg

M.-A. Lemburg wrote:
> Also note that Python will complain loudly when you try to load
> a UCS2 extension in a UCS4 build and vice-versa. We've made sure
> that any extension using the Python Unicode C API has to be built
> for the same UCS version of Python. This is done by using different
> names for the C APIs at the C level.
> 
>> These are not isolated incidents.  The following google searches
>> suggest that a number of people spend time trying to figure out why
>> Python extension modules fail on their linux systems:
>>
>> http://www.google.com/search?q=PyUnicodeUCS4_FromUnicode+undefined+symbol
>> http://www.google.com/search?q=+PyUnicodeUCS2_FromUnicode+undefined+symbol
>> http://www.google.com/search?q=_PyUnicodeUCS2_AsDefaultEncodedString+undefined+symbol
> 
> Perhaps we should add a FAQ entry for these linker errors
> (which are caused by the mentioned C API changes to prevent
> mixing UCS version) ?!

There already is one:

http://www.python.org/doc/faq/extending/#when-importing-module-x-why-do-i-get-undefined-symbol-pyunicodeucs2

I wonder why it doesn't show up in the Google searches.

> Here's a quick way to determine you Python Unicode build type:
> 
> python -c "import sys;print((sys.maxunicode<66000)and'UCS2'or'UCS4')"
> 
> Perhaps we should include this info as well as an 32/64-bit indicator
> and the processor type in the Python startup line:
> 
> # python
> Python 2.6 (r26:66714, Feb  3 2009, 20:49:49, UCS4, 64-bit, x86_64)
> [GCC 4.3.2 [gcc-4_3-branch revision 141291]] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> 
> This would help users find the right binaries to install as
> extension.

Regards,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Sep 28 2009)
>>> Python/Zope Consulting and Support ...http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/


::: Try our new mxODBC.Connect Python Database Interface for free ! 


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 389: argparse - new command line parsing module

2009-09-28 Thread Floris Bruynooghe

On Mon, Sep 28, 2009 at 06:59:45AM +0300, Yuvgoog Greenle wrote:
> -1 for deprecating getopt. getopt is super-simple and especially useful for
> c programmers learning python.
> 
> +1 for argparse.+1 for eventual deprecation of optparse - optparse and
> argparse have a very similar syntax and having both is just
> confusing. tsboapooowtdi

+1 on all of this :-)

It would be a shame to see getopt go but optparse -> argparse seems
logical.


Floris
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] IO module precisions and exception hierarchy

2009-09-28 Thread Pascal Chambon

+-InvalidFileNameError (filepath max lengths, or "? / : " characters
in a windows file name...)

This might be a bit too precise. Unix just has EINVAL, which
covers any kind of invalid parameter, not just file names.

Allright thanks, an InvalidParameter (or similar) exception should do it
better then.

Personally I'd love to see a richer set of exceptions for IO errors,
so long as they can be implemented for all supported platforms and no
information (err number from the os) is lost.

I've been implementing a fake 'file' type [1] for Silverlight which
does IO operations using local browser storage. The use case is for an
online Python tutorial running in the browser [2]. Whilst implementing
the exception behaviour (writing to a file open in read mode, etc) I
considered improving the exception messages as they are very poor -
but decided that being similar to CPython was more important.

Michael

[1]
http://code.google.com/p/trypython/source/browse/trunk/trypython/app/storage.py
and
http://code.google.com/p/trypython/source/browse/trunk/trypython/app/tests/test_storage.py

[2] http://www.trypython.org/

Cool stuff :-)
It's indeed quite unsure at the moment which exceptions it will really
be possible (and relevant) to implement in a cross-platform way... I
guess I should use my own fileio implementation as a playground and a
proof of concept, before we specify anything for CPython.

What happens isn't specified, but in practice (with the current
implementation) the overwriting will happen at the byte level, without
any check for correctness at the character level.

Actually, read+write text streams are implemented quite crudely, and
little testing is done of them. The reason, as you discovered, is that
the semantics are too weak, and it is not obvious how stronger semantics
could look like. People wanting to do sophisticated random reads+writes
over a text file should probably handle the encoding themselves and
access the file at the binary level.

It sounds ok to me, as long as we notify users about this danger (I've
myself just realized about it). Most newcomers may happily open an UTF8
text file, and read/write in it carelessly, without realizing that the
characters they write actually screw up the file...

How about just making IOError = OSError, and introducing your proposed
subclasses? Does the usage of IOError vs OSError have *any* useful
semantics?

I though that OSError dealt with a larger set of errors than IOError,
but after checking the errno codes, it seems that they're all more or
less related to IO problems (if we include interprocess communication in
I/O). So theoretically, IOErrors and OSErrors might be merged. Note that
in this case, windowsErrors would have to become children of
EnvironmentError, because windows error code really seem to go farther
than io errors (they deal with recursion limits, thousands of PC
parameters...).
The legacy is so heavy that OSError would have to remain as is, I think,
but we might simply forget it in new io modules, and concentrate on an
IOError hierarchy to provide all the info needed by the developer.

Some of the error messages are truly awful though as things stand,
especially for someone new to Python. Try to read from a file handle
opened in read mode for example: IOError: [Errno 9] Bad file descriptor

Subdividing the IOError exception won't help with
that, because all you have to go on when deciding
which exception to raise is the error code returned
by the OS. If the same error code results from a
bunch of different things, there's not much Python
can do to sort them out.

Well, you don't only have the error number, you also have the context of
this exception. IOErrors subclasses would particularly be useful in a
"high level" IO contect, when each single method can issue lots of
system calls (to check the file, lock it, edit it...). If the error is
raised during your locking operation, you can decide to sort it as
"LockingError" even if the error code provided might appear in several
different situations.

Regards,
Pascal

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] IO module precisions and exception hierarchy

2009-09-28 Thread Michael Foord


Greg Ewing wrote:

Michael Foord wrote:

Some of the error messages are truly awful though as things stand, 
especially for someone new to Python. Try to read from a file handle 
opened in read mode for example: IOError: [Errno 9] Bad file descriptor


Subdividing the IOError exception won't help with
that, because all you have to go on when deciding
which exception to raise is the error code returned
by the OS. If the same error code results from a
bunch of different things, there's not much Python
can do to sort them out.



Well, the file type knows what mode it is opened in so catching these 
errors is really not difficult at all.


Michael



The error messages produced for the various error
codes could perhaps be improved, but that's an
orthogonal issue.




--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 389: argparse - new command line parsing module

2009-09-28 Thread Neal Becker

If the plan is to migrate from optparse to argparse, this could be made a 
bit easier.  If it weren't for the fact that some names are different in 
argparse than optparse, I believe many optparse usages could port with no 
change.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 389: argparse - new command line parsing module

2009-09-28 Thread Michael Foord


Neal Becker wrote:
If the plan is to migrate from optparse to argparse, this could be made a 
bit easier.  If it weren't for the fact that some names are different in 
argparse than optparse, I believe many optparse usages could port with no 
change.
  


The new names in argparse fit with the fact that argparse has a 
different 'rationale' than optparse. What about an optparse 
compatibility layer to make porting easier?


Although it can't cover the whole optparse API (as explained in the PEP) 
it could perhaps cover most 'normal' usage?


Michael


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
  



--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

90 matches

Mail list logo